ECS `RunTask` needs public IP to access env file
Learning how to use ECS tasks to run some cron jobs has been an opaque journey to say the least.
I knew my
TaskDefinition was fine, because my server was running, but I wanted to use that same definition with a different container command and schedule its execution for housekeeping tasks.
I started with creating an
EventBridge schedule, which seemed straight forward enough. But as soon as I created it I was puzzled that there didn't seem to be a way to trigger it for testing nor was there any kind of information on whether or not the last invocation had happened and succeeded.
Checking the next day, it didn't seem like the cron job had run and there was nothing in the logs, since that only emits if it actually got to the running state. After some digging, I found the
RunTask event in
CloudTrail, but trying to look up the task, it was
MISSING. Apparently completed tasks are only stored for a short period.
Trying to reduce the surface area of this problem, I decided to run the task with command override directly in ECS, which resulted in:
I currently keep my task environment vars in a file in a private S3 bucket. Could have taken this as a hint to switch to
ParameterStore instead, but I was determined to figure this out, since it worked for the server.
I should have just googled that error right then and there, since when I eventually did that, I found out what took me way too long to devise on my own. Instead, I ran the task a number of times, starting with the server configuration and backing out things I didn't think were needed one at a time until I discovered that the task needs a public IP to access the env file , i.e.:
Oh, and as a final source of confusion, when trying to use a
ENABLED was not an option in the console. Only after switching to
FARGATE did that become available.