Okay, take one of those, the NPC AI, to be able to have an NPC Crew that is going to be a member of the crew you need to pretty much understand all the possible inputs and define an action for them.
I don't actually think this needs to be as difficult as it's being made to seem. Our NPC fighter pilots already do the things NPC crew would need to do.
Let's look at the individual roles:
Fighter Pilot - already covered - done.
Gunner - Fighter pilot AI already knows how to operate a ship and aim and shoot, so this role would only need to be "lightened", as no power management would be required. They would need only simple orders:
1. Aggressive - fire on any valid target in range (Fire at Will)
2. Focused - fire only on the same target as the pilot (Fire on my target)
3. Defensive - Fire only on targets firing on mother ship.
4. Stand By - Don't fire at all
Engineer - purely a power-management role, which again, the NPC fighter pilots already do. A simplified order set would really be all that is needed.
1. Aggressive - prioritize power to weapons (maintain full power to weapons as long as other capacitors are not drained, shift power when other capacitors are less than 10%)
2. Defensive - prioritize power to Systems. (maintain full power to Systems as long as other capacitors are not drained, shift power when other capacitors are less than 10%)
3. Evasive - Prioritize power to Engines. (maintain full power to Engines as long as other capacitors are not drained, shift power when other capacitors are less than 10%)
4. Balanced - Prioritize balanced power. (maintain balanced power as long as other capacitors are not drained, shift power when other capacitors are less than 10%)
They would need only to shift power around when necessary, and then would do so based on the priority orders given.
Science Officer - Not really a current role, but would function much like a Gunner, only operating ADS and DSS
Commando - No idea what this role might entail, as we're stuck in our ships and paralyzed from the waist down.
But I do think these roles could be readily adapted from the current NPC fighter pilot AI routines.
As for how to recruit them...
Crew Lounge would be a good start. Rescued from Occupied Escape Pods would be a great addition.
Recruited as part of a mission or mission chain would also be a great addition.
While you do raise a number of valid questions, most of these are already answered by the simple presence of NPC fighter pilots - as for how this fits with the vision of the game - mechanical crew and roles were something initially pitched, with a fair amount of detail given about their operation, as well as biological crew. It's rather old-hat, and just has not yet been implemented.