Greetings, if you are interested in AX (specifically wing combat), or want to understand how the infamous Invincible Heart bug happens and how to work around it, this post is for you. If not, you will likely be bored to tears.
Disclaimer
All information presented here was gathered from thousands of reported cases, first-hard experience, hundreds of hours of dedicated testing, and parsing of the game's freely-available network logs. No data-mining or memory-reading was used, even though it would've probably sped up the process. I do not have access to any privileged information, and had to piece the picture together from what I had access to, so the result may be incorrect on some or all levels. I do not pretend to know the absolute truth about the issue, or the way to fix it. I am simply collecting all available information about the bug in one place, for the benefit of AX community.
Part 1 - Background
The Invincible Heart bug takes its name from what it appears to be - exerted thargoid hearts not taking any damage in some circumstances. The reality however is a bit more complicated than that, but we will get to it in due time. The earliest recorded mention of the bug (known to me) dates back to May 2018, which in turn notes that the issue was already well-know before 3.0 update. Based on my personal recollection, that post was made soon after the concept of the issue had solidified in the mind of the community, when we understood that something was indeed wrong and had done first test to reproduce it. Once the new-at-the-time issue tracker was launched, the bug report was ported there and appended with new information. Since then, it remains one of the top-voted issues on the tracker.
FDev has deployed several patches over the years attempting to fix or mitigate the issue, but so far they only had marginal effect on its frequency, so the issue report had been re-opened many times. It is the hope of all of the AX community that this issue will finally be fixed with the launch of Odyssey and promised refresh of the game's codebase.
Part 2 - When it happens
Almost any time there are 2+ player in the instance (wing or multicrew) with a Thargoid Interceptor. Thargoid Scout do not have hearts so they are not affected by this issue. There is no clear correlation between frequency of the bug happening and any connection aspect we were able to test for. People connecting literally across the globe (tested Europe-USA, Europe-Australia, Australia-USA and Europe-Asia) can have perfect fights, while people sitting in the same room could have every heart become "invincible". Instances of 8+ people could have minimal issues, while a wing of two may be unable to kill any hearts.
Ultimately, we were unable to determine the exact circumstances with the limited tools we have access to. It is most likely not completely random (nothing in programming is), but relies on some factors we are not able to control. We do, however, have a set of recommendations that, we are reasonably certain, will reduce the chances of the bug occurring. Those are be listed in part 5.
Part 3 - Why and how it happens
At its core the Heart Bug is (most likely) a networking/instancing issue, with a bit of spaghetti-code on the side. For almost 2 years after its discovery, we assumed that the exerted heart just failed to register damage from some players. Every time the bug happens there is always at least one player (colloquially know as "the chosen one") that can continue to deal damage to the heart, so that player is suddenly responsible to carry the fight and take out the heart essentially solo. This is, however not exactly true. After FDev enabled additional logging for the interceptor combat, we were able to dig into the logs and figure out how the exertion status is communicated across clients.
I will assume you understand the basics of peer-to-peer (p2p) network scheme. If not, I encourage you to read about it. Elite is at heart, a p2p game, with only select transactions happening with FDev servers, and most of the game being entirely client-to-client. Every object in the instance has an "owner" client, that is responsible for managing it and communicating changes happening with it to all other clients in the instance. This means that only one client will always definitively know what is happening with that object, and the rest have to get that information from it. Here lies the main potential for any de-synchronization that can happen if the connection gets somehow interrupted. Simplifying the whole thing, when non-owner client does anything with the object he doesn't own, he sends a message to the owner of the object that essentially reads "I did such and such, please process it". In most cases, what both sides see on their end is the same so the response makes sense. In cases of de-sync however, this creates some weird results.
One of these client-to-client things that need to be synchronized is the exertion status of the hearts. This is done by a HeartManager subsystem inside the game. The "owner" client broadcasts the change in status to all other clients, and logs it like this:
The other clients will receive the message (after some delay) and log it like this:
As you can see, the ID of the exerted heart (which is always in the ballpark of 30-40) is passed inside the message.
Sometimes, however, the following happens:
First client
Second client
As you can see there are some inconsistencies in these entries. Those of you with some programming knowledge already realized that 4294967295 is the exact maximum value that can be stored in an unsigned 32bit integer variable. This strongly indicates that an overflow happens somewhere on the owner client and the wrong value is send out to other clients. This results in the wrong heart being shown as exerted on non-owner clients, as there is no heart with ID of -1. The exact mechanism of how a heart is chosen to be exerted is obviously not known.
There is, however, one more aspect of this mess we realized all too late - you can still deal damage to the "actually exerted" heart, but with a caveat. AXI have done extensive testing of this and determined that if you see that you cannot damage the heart you see as exerted, you can try to shoot other hearts (which are not exerted on your screen) and if you find the "true" heart, the damage you do to it will be reflected on the heart you see as exerted, which is not necessarily the same heart as the on the owner client. Take a minute to wrap your head around this.
This creates a very confusing situation that deteriorates the longer the fight goes on. If the heart that non-owner client sees as exerted is destroyed it creates a permanent desync that can lead to complete instance splitting. Reason for this is simple - since that heart is still intact on the owner client, it can become exerted later. In this case, even if the bug does not occur, non-owner client will not be able to damage that heart at all, as on their side it literally does not exist and the shots will go straight through it. This is especially obvious during later stages of a Hydra fight, when most of the petals are destroyed.
I hope this explanation is clear enough.
Part 4 - Why it matters
During my time as AXI Overseer, I was many times in the situation where I had to explain the appeal of AX combat, and why it is, in my opinion, the most engaging part of the game. But at the same time I had to explain all the bugs and issues unique to it, that they were not fixed over the entire existence of our community, and that they will, most likely, remain unfixed for a long time to come. As you can imagine it is very taxing - understanding that your favorite part of the game is fundamentally broken, and developers, for whatever reason, are unable or unwilling to fix it. Many old-time AX veterans and mentors crumble under that realization, frequently devolving into cynicism and disdain towards the developers. This, in turn, does not help to fix the issue, as verbal abuse does not contain any useful feedback. The longer it goes unfixed, the more animosity it will create.
Many, if not most, players coming to AXI want to fight thargoids with their friends and are hit with the realization that simply having another person in the instance is actively detrimental to the experience, sometimes to the point of unplayability. It is very heartbreaking being forced to explain the situation to every new arrival and see many lose interest due to it. We would prefer to not be forced to say negative things about our chosen playstyle, but lying about the situation is not an option.
It is also clear that thargoid interceptors, especially the harder ones, were designed with multiplayer in mind, which makes the situation even more bitterly-ironic. While AX CZs were present, many players experimented with support builds for wing-clearing, but ran into this bug. If the support player foregoes the usual gauss-only loadout in favor of AX MCs or other support weapons, they lose a significant chunk of DPS. During the fight they then may become "the chosen one", forced to be the one to take down the heart with an inferior loadout, potentially leading to the instance being unwinnable. This has a very significant effect on the AX meta, forcing every single ship in the instance to be able to take out hearts efficiently, or risk complete failure. This alone invalidates many creative builds that would otherwise see wide use, and unnecessarily limits the potential for cooperation between players.
The effectively random occurrence of the bug also prohibits any attempts at balancing wing AX payouts, as the fight time is essentially at the mercy of the networking. If the bug does not happen, the fight takes less time than an equivalent solo kill, but if it does, it can be drawn out many times over. When planning events, we have to estimate the fight length as at least twice as long as the typical solo kill.
Part 5 - How to avoid it
In short - there is no sure way to avoid it, except playing alone. Your connection may just so happen to have the required parameters to never have to worry about this bug, but absolute majority of people will encounter it sooner of later. Even if there was a way, it is simply impossible for us to do the testing necessary to find it, since we are very limited in tools and information. There are however some recommendations that, according to reasonably-reliable reports, help lower the chance of the issue happening to you.
- Setting up port-forwarding for the game. This is a minor, but also very easy method of smoothing your wing experience. It's effectiveness varies wildly between people, so it might do it for you, or have no noticeable effect.
- Using wing navlock when dropping into the instance. Many reliable reports suggest that doing this significantly decreases the chances of the bug happening during the fight, so long as nobody leaves or enters the instance after the initial drop.
- Fighting in the mixed (scouts+interceptor) instances. Some reports suggest that the heart bug never happens in such instances if all players are in the instance before the interceptor jumps in. Attacked megaship instances also work for (probably) the same reason.
If you have counter-examples to the above methods (with video evidence) - feel free to post them here. Any other questions or additions are also welcome.
Disclaimer
All information presented here was gathered from thousands of reported cases, first-hard experience, hundreds of hours of dedicated testing, and parsing of the game's freely-available network logs. No data-mining or memory-reading was used, even though it would've probably sped up the process. I do not have access to any privileged information, and had to piece the picture together from what I had access to, so the result may be incorrect on some or all levels. I do not pretend to know the absolute truth about the issue, or the way to fix it. I am simply collecting all available information about the bug in one place, for the benefit of AX community.
Part 1 - Background
The Invincible Heart bug takes its name from what it appears to be - exerted thargoid hearts not taking any damage in some circumstances. The reality however is a bit more complicated than that, but we will get to it in due time. The earliest recorded mention of the bug (known to me) dates back to May 2018, which in turn notes that the issue was already well-know before 3.0 update. Based on my personal recollection, that post was made soon after the concept of the issue had solidified in the mind of the community, when we understood that something was indeed wrong and had done first test to reproduce it. Once the new-at-the-time issue tracker was launched, the bug report was ported there and appended with new information. Since then, it remains one of the top-voted issues on the tracker.
FDev has deployed several patches over the years attempting to fix or mitigate the issue, but so far they only had marginal effect on its frequency, so the issue report had been re-opened many times. It is the hope of all of the AX community that this issue will finally be fixed with the launch of Odyssey and promised refresh of the game's codebase.
Part 2 - When it happens
Almost any time there are 2+ player in the instance (wing or multicrew) with a Thargoid Interceptor. Thargoid Scout do not have hearts so they are not affected by this issue. There is no clear correlation between frequency of the bug happening and any connection aspect we were able to test for. People connecting literally across the globe (tested Europe-USA, Europe-Australia, Australia-USA and Europe-Asia) can have perfect fights, while people sitting in the same room could have every heart become "invincible". Instances of 8+ people could have minimal issues, while a wing of two may be unable to kill any hearts.
Ultimately, we were unable to determine the exact circumstances with the limited tools we have access to. It is most likely not completely random (nothing in programming is), but relies on some factors we are not able to control. We do, however, have a set of recommendations that, we are reasonably certain, will reduce the chances of the bug occurring. Those are be listed in part 5.
Part 3 - Why and how it happens
At its core the Heart Bug is (most likely) a networking/instancing issue, with a bit of spaghetti-code on the side. For almost 2 years after its discovery, we assumed that the exerted heart just failed to register damage from some players. Every time the bug happens there is always at least one player (colloquially know as "the chosen one") that can continue to deal damage to the heart, so that player is suddenly responsible to carry the fight and take out the heart essentially solo. This is, however not exactly true. After FDev enabled additional logging for the interceptor combat, we were able to dig into the logs and figure out how the exertion status is communicated across clients.
I will assume you understand the basics of peer-to-peer (p2p) network scheme. If not, I encourage you to read about it. Elite is at heart, a p2p game, with only select transactions happening with FDev servers, and most of the game being entirely client-to-client. Every object in the instance has an "owner" client, that is responsible for managing it and communicating changes happening with it to all other clients in the instance. This means that only one client will always definitively know what is happening with that object, and the rest have to get that information from it. Here lies the main potential for any de-synchronization that can happen if the connection gets somehow interrupted. Simplifying the whole thing, when non-owner client does anything with the object he doesn't own, he sends a message to the owner of the object that essentially reads "I did such and such, please process it". In most cases, what both sides see on their end is the same so the response makes sense. In cases of de-sync however, this creates some weird results.
One of these client-to-client things that need to be synchronized is the exertion status of the hearts. This is done by a HeartManager subsystem inside the game. The "owner" client broadcasts the change in status to all other clients, and logs it like this:
{12:44:08GMT 756.532s} HeartManager - Authority SetExertedHeartSlotIndex: 36 (36)
{12:44:08GMT 756.532s} HeartManager - Authority SetExerted True. Heart: 36The other clients will receive the message (after some delay) and log it like this:
{12:44:11GMT 251.804s} HeartManager - NonAuthority SetExertedHeartSlotIndex: 36 (36)
{12:44:11GMT 251.804s} HeartManager - Non-Authority SetExerted True. Heart: 36As you can see, the ID of the exerted heart (which is always in the ballpark of 30-40) is passed inside the message.
Sometimes, however, the following happens:
First client
{12:46:48GMT 916.430s} HeartManager - Authority SetExertedHeartSlotIndex: 4294967295 (4294967295)Second client
{12:46:51GMT 411.875s} HeartManager - NonAuthority SetExertedHeartSlotIndex: 4294967295 (4294967295)
{12:46:51GMT 411.875s} HeartManager - Non-Authority SetExerted False. Heart: -1As you can see there are some inconsistencies in these entries. Those of you with some programming knowledge already realized that 4294967295 is the exact maximum value that can be stored in an unsigned 32bit integer variable. This strongly indicates that an overflow happens somewhere on the owner client and the wrong value is send out to other clients. This results in the wrong heart being shown as exerted on non-owner clients, as there is no heart with ID of -1. The exact mechanism of how a heart is chosen to be exerted is obviously not known.
There is, however, one more aspect of this mess we realized all too late - you can still deal damage to the "actually exerted" heart, but with a caveat. AXI have done extensive testing of this and determined that if you see that you cannot damage the heart you see as exerted, you can try to shoot other hearts (which are not exerted on your screen) and if you find the "true" heart, the damage you do to it will be reflected on the heart you see as exerted, which is not necessarily the same heart as the on the owner client. Take a minute to wrap your head around this.
This creates a very confusing situation that deteriorates the longer the fight goes on. If the heart that non-owner client sees as exerted is destroyed it creates a permanent desync that can lead to complete instance splitting. Reason for this is simple - since that heart is still intact on the owner client, it can become exerted later. In this case, even if the bug does not occur, non-owner client will not be able to damage that heart at all, as on their side it literally does not exist and the shots will go straight through it. This is especially obvious during later stages of a Hydra fight, when most of the petals are destroyed.
I hope this explanation is clear enough.
Part 4 - Why it matters
During my time as AXI Overseer, I was many times in the situation where I had to explain the appeal of AX combat, and why it is, in my opinion, the most engaging part of the game. But at the same time I had to explain all the bugs and issues unique to it, that they were not fixed over the entire existence of our community, and that they will, most likely, remain unfixed for a long time to come. As you can imagine it is very taxing - understanding that your favorite part of the game is fundamentally broken, and developers, for whatever reason, are unable or unwilling to fix it. Many old-time AX veterans and mentors crumble under that realization, frequently devolving into cynicism and disdain towards the developers. This, in turn, does not help to fix the issue, as verbal abuse does not contain any useful feedback. The longer it goes unfixed, the more animosity it will create.
Many, if not most, players coming to AXI want to fight thargoids with their friends and are hit with the realization that simply having another person in the instance is actively detrimental to the experience, sometimes to the point of unplayability. It is very heartbreaking being forced to explain the situation to every new arrival and see many lose interest due to it. We would prefer to not be forced to say negative things about our chosen playstyle, but lying about the situation is not an option.
It is also clear that thargoid interceptors, especially the harder ones, were designed with multiplayer in mind, which makes the situation even more bitterly-ironic. While AX CZs were present, many players experimented with support builds for wing-clearing, but ran into this bug. If the support player foregoes the usual gauss-only loadout in favor of AX MCs or other support weapons, they lose a significant chunk of DPS. During the fight they then may become "the chosen one", forced to be the one to take down the heart with an inferior loadout, potentially leading to the instance being unwinnable. This has a very significant effect on the AX meta, forcing every single ship in the instance to be able to take out hearts efficiently, or risk complete failure. This alone invalidates many creative builds that would otherwise see wide use, and unnecessarily limits the potential for cooperation between players.
The effectively random occurrence of the bug also prohibits any attempts at balancing wing AX payouts, as the fight time is essentially at the mercy of the networking. If the bug does not happen, the fight takes less time than an equivalent solo kill, but if it does, it can be drawn out many times over. When planning events, we have to estimate the fight length as at least twice as long as the typical solo kill.
Part 5 - How to avoid it
In short - there is no sure way to avoid it, except playing alone. Your connection may just so happen to have the required parameters to never have to worry about this bug, but absolute majority of people will encounter it sooner of later. Even if there was a way, it is simply impossible for us to do the testing necessary to find it, since we are very limited in tools and information. There are however some recommendations that, according to reasonably-reliable reports, help lower the chance of the issue happening to you.
- Setting up port-forwarding for the game. This is a minor, but also very easy method of smoothing your wing experience. It's effectiveness varies wildly between people, so it might do it for you, or have no noticeable effect.
- Using wing navlock when dropping into the instance. Many reliable reports suggest that doing this significantly decreases the chances of the bug happening during the fight, so long as nobody leaves or enters the instance after the initial drop.
- Fighting in the mixed (scouts+interceptor) instances. Some reports suggest that the heart bug never happens in such instances if all players are in the instance before the interceptor jumps in. Attacked megaship instances also work for (probably) the same reason.
If you have counter-examples to the above methods (with video evidence) - feel free to post them here. Any other questions or additions are also welcome.
Last edited:
