Yeah, a brown dwarf isn't actually brown, the name is to differentiate it from a red dwarf. Since brown is not even a color, but merely a very dark red, it makes sense to associate it with objects even smaller and dimmer than red dwarves.
I think I've also read somewhere that brown dwarves might indeed be dark purple, sadly I have no idea where and therefore cannot provide a link.