Build server is unreachable: HTTP error: 502 - Bad Gateway: undefined

OttoFMS
1

I tried doing a deployment overnight. The solution consists of 8 files. The first 4 were “built” successfully (does a build simply consist of creating a clone?), but the 5th one failed.

I’ve run this deployment (with the same set of 8 files) successfully once before.

Is there some way I can find out what went wrong, so I can figure out how to prevent it from happening again?

Here’s the “detailed log” from the portal:

Phase File Message Duration
scheduled - deployment scheduled 00:00.000
queued - Deployment queued 00:00.000
starting - Starting deployment: Deploy to PROD 00:00.000
starting - 8 files 00:00.008
closing - Starting to close files 00:02.007
closing - Files closed successfully 00:18.208
building - Sending just in time build request to source server 00:18.217
building - Just in time build started on source server 00:19.207
building - Build running, completed 1 of 8 files 00:56.380
building - Build running, completed 2 of 8 files 01:04.560
building - Build running, completed 3 of 8 files 01:20.925
building - Build running, completed 4 of 8 files 02:10.071
building - Build server is unreachable: HTTP error: 502 - Bad Gateway: undefined 02:27.966
opening - Starting to open files 02:29.493
opening - Files opened successfully 02:39.558
post-deployment script - Post-deployment script not run since deployment failed 02:39.566
done - Deployment process failed. Original files are unmodified. 02:39.573
post-deployment steps - Deleting schedule 02:39.612
2

Hi Mislav,

What operating system are you using? Is it Windows?

Thanks

Todd

3

Yes, Windows. Sorry, should have mentioned that previously.

Windows Server 2022 Datacenter
FMS Version: 20.3.2
OttoFMS Version: 4.2.3.240313464

4

Hi Mislav,

Thanks for confirming that. I suspected as much. :frowning:

Could you send us the otto-info.logs from both servers? You can use a direct message or email to support@proofgeist.com if you want.

Thanks

Todd

5

Hey,

I just noticed that you are on a slightly older version of OttoFMS. The current version has a fix in it that should make this problem go away.

Read this for more info for updating

6

Thanks, I’ll update and try to deploy again.

Do you still want the otto-info.logs from both servers?

7

No.

Upgrade your servers, and try it again.

Let us know what happens

Todd

8

I got a different kind of error this time: Deployment crashed.

Here’s the detailed log from the Otto portal:

Phase File Message Duration
scheduled - deployment scheduled 00:00.000
queued - Deployment queued 00:00.000
starting - Starting deployment: Amps Deploy to PROD 00:00.000
starting - 8 files 00:00.008
unknown phase - Deployment crashed 00:05.041

Versions from both DEV and PROD:
FMS Version: 20.3.2
OttoFMS Version: 4.3.3.240429424

I’ll DM you the Otto logs from both servers.

9

What are the server stats? How many CPU’s and how much RAM?

Todd

13

Mislav,

It looks like the deployment was triggered twice in a row. That shouldn’t have caused an issue. It should have just ignored the second attempt. But instead it just aborted the deployment. We’ll look into that.

It’s possible that a double click got sent through. We’ll also look at that.

But in the meantime, you may want to just try again.

Thanks

Todd

15

Thanks Todd. I’ll give it another try tonight and will report back how it goes.

Server specs:
PROD: m6i.2xlarge … 8 vCPUs, 32 GB
DEV: t3.large … 2 vCPUs, 8 GB

16

Last night’s deployment worked without any problems. Thanks for the support you’ve provided to get me there.

In the previous two attempts, the glitch happened early on, before DMT. I was talking to a colleague of mine, and he reported a similar experience. In his case, he kicks off the deployment ad hoc and is able to monitor it during the initial phase, so it’s no big deal for him to manually retry it.

In my case, I have to schedule my deployment for overnight, because of the work hours that my client maintains, so it’s not convenient for me to retry it manually. Something to consider for you guys is to build in some retry logic if the deployment fails in those initial steps.

Thanks again for making this great tool available to the community.

1 Like
17

Hi Mislav,

If your colleague could send us the logs from deployment server when that happens, that would be helpful. The more information we can get the better.

Thanks

Todd

18

Hi Todd,

I’ve sent the logs from my server to the support email.

Thanks!

-Emilio

1 Like