Grab an external hard drive (or a USB flash drive of 16GB or more) and copy the game files from the install location to it. The default install location on a non-Steam version should be something like C:\Program Files\Frontier\.
Download the launcher and install it on your new PC. DON'T INSTALL THE GAME THROUGH THE LAUNCHER.
Copy the directory back over to the new location. If you leave everything alone, it should be the same as the location on the old PC. Keep a note of where you install it if you choose a custom directory. Launch the launcher, log in and install the versions you had installed on your old machine. Instead of downloading the files again (which can be painfully slow,) the launcher will download a much smaller manifest file and validate the game files. You'll be ready to go in 15 minutes, less if you have installed the game to an SSD.
You can do the same with the Steam version, although the install locations are obviously going to be different.
Your key binds and other settings are stored in C:\Users\<your username>\AppData\Local\Frontier Developments\Elite Dangerous\Options Copy this folder from your old machine and drop it into the same place in the new one. Select "yes" if it asks you if you want to overwrite. Your control binds will be exactly as you left them on your old machine, along with graphics and sound options.