Sometimes, spot fleets get terminated; this termination is a safety net for not getting surprise bills. The component responsible to take care of that is known as Resource Tracker (RT). We have prepared documentation to explain how RT works and to serve as reference to help with troubleshooting why spot fleets were terminated.
Here's the detailed information about the resource tracker:
If there is a "Fleet Error" status in red in the bottom right side of Deadline Monitor , it means that Resource Tracker has marked the fleet as unhealthy. Click it, it will show a popup where you will find the Spot Fleet ID link. Go to the link and make sure no instances are still running under the fleet ID. You will see status of Spot Fleet if its marked as terminated which means there are no instances running. Now come back to the popup on Deadline Monitor and add your Access and Secret Key for the respected IAM user you have created on AWS to fix the issue.
If you are troubleshooting, you can jump straight to the troubleshooting section of the Resource Tracker docs. If you are unable to solve this issue with the troubleshooting guides, or if it keeps happening, please reach out to us by cutting us a ticket or calling us if it is urgent, contact information is here. Don't forget to share the logs from terminated fleet.
Comments
0 comments
Article is closed for comments.