Wall of text alert, for those who say TL;DR. The site has been slow and has some issues, I've already improved quite a lot of things and am working to improve things further.
So I'm aware there has been some slowness over the last week or two (Especially yesterday when I was trying a few things), after a fair amount of work yesterday I believe I have mitigated most of the issues. Listed below are a few of the issues related to speed, along with a few other issues which have come to light.
1. Loading 1.2million Neutron Stars into the plotter was causing a few issues with the database every 10 minutes or so causing routes to take up to 5+ minutes to generate, I've added a second caching layer which is populated every 10 minutes from the database, routes are now taking a few seconds to generate once more.
2. The website was checking for completed routes every second, causing a small amount of load on the api. This wasn't causing major issues but was adding to the problems when the site was under load. I've altered this so it now throttles down gradually. This means that it'll check for completed routes like so. 1 second, 2 seconds, 4 seconds, 8 seconds, 16 seconds, 30 seconds (it will then remain at 30 seconds). This means that in the worst case scenario routes may seem like they are taking longer (as the site won't check for up to 30 seconds if the route has been completed), but will still generate. Under normal conditions people hopefully won't notice.
3. An upgrade to Elasticsearch caused some of my queries to underperform for the Road to Riches (in some cases taking up to 30 seconds to generate even when not under load). Some aggressive optimisation and tweaks have brought this back down to 1-2 seconds.
4. A while back I optimised my Elasticsearch import (what causes the Road to Riches and the Systems and Bodies searches to be up to date), this brought the time taken to import the roughly 100GB of data to go from 12+ hours to 4 in the best case scenario. The same upgrade to Elasticsearch along with the increased import time caused the entire backend machine to slow down quite severely. I have throttled this import process severely such that it will once again take quite a long time but should not impact the live site. This isn't a long term solution but should make sure that the data is up to date whilst still allowing the site to work.
5. When the backend was under several of the API calls which gather data to display the site (the range sliders, dropdown boxes etc) could take a long time to generate. Especially on the bodies page which has 15-20 of those that could cause some quite awkward display issues. The other fixes I have done should alleviate this issue for the meantime.
6. Not backend related, but the Neutron Star map on the site now brings almost any browser to a halt (1.2million stars is a lot). I have an improved version of that map which needs some final polish. It should be substantially more performant and will also allow partial maps to be displayed whilst the data to display is being loaded. I'm not sure when I'll get time to finish that off however.
7. The user interface has atrophied quite a lot with recent additions and looks quite ugly in a lot of places.
8. Neutron routes generated around the edges of the galaxy often plot people through the voids around the spiral arms.
9. New Neutron Stars were not being picked up by the plotter. This was due to a simple coding error on my part and has now been fixed. This has meant the site was only using 700,000 Neutron Stars rather than the 1.2million it was supposed to be using. I've reprocessed things and filled in the missing data so routes should now be substantially better.
10. When generating an A to B Road to Riches you often end up with a large number of stops at the start and the finish. This is due to the shape used to pick out viable systems (a cylinder), so you can end up going 90 degrees to the way you want for quite some time. I'd like to do a second processing step to even this out a little. As such I'm going to attempt to change the shape from a plain Cylinder to a shorter Cylinder with two half spheres on either end (like a sausage shape). This should mean that when people regenerate a route from a new place it won't immediately take them directly away from where they intended to go.
11. The results URLS for the plotter are very long. I'm in two minds about this so I wouldn't mind some feedback. I don't particularly mind the long URLs and they show you exactly what the route was without having to load the page. It would be relatively easy for me to save the parameters for each generated route in the database and have them loaded. It would require a small extra load step when loading but should not cause much extra time. If people really prefer this I'll look at adding it.
With regards to Elasticsearch I have several other options I'm considering for making sure the Elasticsearch instance (which, to be honest is much too small for the amount of data I'm currently throwing at it) remains performant. These include
1. Generating the index on a different machine then transferring it over. The downside of this would be having to bring down the index to refresh.
2. Keeping the index up to date on the fly rather than a full import every day, then refreshing the entire index once a week. The downside of this is that incorrect data which needs to be removed will remain in the index for longer, potentially generating invalid or incorrect routes.
3. Increasing the hardware so the index has more processing power. The downside of this is obviously cost.
On the other issues I am hoping to get some time to do the following.
1. Routes through the void. I'm hoping to generate a mask for the Neutron plotter which will avoid and route around the voids in the galaxy. Now that the data we have on the galaxy is substantially more complete (though still only 0.001% of the whole), I should be able to generate a mask and adjust the plotter to use a combined A*/Geographic method to route people around the voids. This will likely involve a preprocessing step to break the route down into parts, so that when you generate a route it will automatically break it down into sections to route around the voids, then generate individual routes inbetween those sections. Since these sections are fairly (read: completely) static a lot of this precalculation can be done beforehand.
2. Search user interface ugliness/complicatedness. Because of the sheer number of fields on the search pages there is a lot of space taken up by fields people probably don't use for every search (I imagine most people probably search on 2-5 fields). Rather than have all of those fields available all of the time I'm probably going to split the form into 2 parts. A list of fields and the actual fields. This technique is related to faceted search and you can see something similar to what I'm aiming for here (
https://blog.linkedin.com/2009/12/14/linkedin-faceted-search ). Because some of the numbers on those fields are quite long, I'm still intending to have the fields displayed on the main page, but instead of displaying the search form at the same time as the results, I'm intending to have the results page on it's own, with a small summary of the search performed at the top. This will mean people don't have to scroll down as far in order to see their results.
3. Route results pages can break quite severely on smaller windows/mobile devices. Due to the number of fields I show on the route results pages (especially on the road to riches and trade planner) the tables get quite drastically cut off. I've already added a "hide the search" button to allow people to see only the results rather than the search. I'm likely going to take this further and stop displaying the forms on the results pages at all, instead displaying a summary of the route). In both this and the previous point there will be no load time to bring up the search again because of how the site is built. There will be a link/tab which allows people to switch between the form and the results instantly (it's loaded and will still have the data, just hidden).
All of this is related to problems/issues with the site. I am still intending to improve things further and add extra features like.
1. Fuel Stops. Probably the biggest remaining ask from people. This will make the form more complicated as I will need to know the fuel tank size and a few other details in order to calculate when you need to refuel.
2. Non primary Neutron Stars. There are occasionally Binary (or greater) systems with Neutron Stars which are close to the main star which are currently not used, these systems are actually often ideal as they can allow a refuel and supercharge at the same time (Also systems which have a main neutron star and a refuelable star close by). These are relatively rare and probably won't come up much but I'd like to do some more investigation.
3. Stations search. Self explanatory.
4. An A to B route planner. Just a generic, give me an example route from A to B so I can see how far away I am roughly without logging into the game.
5. Visit these places route planner. Often people have a list of places they'd like to visit and want to do them in an optimal order to minimise time. The code for this is essentially identical to the Road to Riches, I just need to make it a little more generic.
The last thing I'd like to say is thank you to everyone who has been donating to the site to help out and also to those people who have emailed me to report bugs or other issues. I try to stay on top of the site performance and pick out bugs but people emailing me to alert me (or ping me on the EDCD discord or fuel rats IRC) to make sure I know about things is not a bad idea.