I have a N. Wennerae an N. Curacaoensis so most of what I say is from what I've read.....
I believe the most interactive would be Peacock mantis, G. Smithii, O. Havanensis, O. Brevirostris, O. Latirostris from Roys List.
You could fit a O Scyllarus in a 40gal.
I have a 1 an a half inch curacaoensis in a 3 gallon right now, its about equivalent in space to my 3" Wenn. in its 10gallon, of course once it gets bigger I have another 10gal for it. My Curaocaoensis is active and comes out an moves alot but not interactive as in comes up to me like maybe the smaller O. Havanensis, Latirostris, or Brevirostris probably would.
I would house any mantis that didn't get over 3" in the fluval edge.(Sounds really appealing, love the looks of those tanks.

)
Out of the two I have it would be the Curacaoensis because it wanders the tank aaaalooot more then the Wenn. I actually think the Wennerae hates me, its always stacking empty shells/sand up high so I can't see him and he can't see me. But I think my favorite mantis of all would be the G. Smithii as its what you want when you want the "interactive" in a mantis shrimp.
(again mostly my opinion, hope some of it helps)
oh, and as I'm typing, my N. Wenn is just staring at me, It knows what I've been saying!......