Don't copy external container files

It would be great to have a feature to exclude external container files when creating builds or update/replace deployments.

I couldn’t find a place to exclude them (let me know if I’m missing something), and not realizing they were being copied ended up filling up my disk.

We want to be able to copy files from production to staging after a migration is complete and local tweaks have been completed, but we almost never need the container data and including it can be huge amount of overhead.

I am not sure I understand containers are not transferred in migrations?

Hi Ken

Welcome to the community :slight_smile:

Currently, in order to make a copy without closing the file, OttoFMS runs a backup, and because that backup is not part of standard set it moves the containers into the file.

You can change this behavior by moving your data bases to one of the additional Database folders, and setting up an additional container data folder, and finally disabling backups on that folder. See image below.

We are thinking of letting you just get a copy of the file with out using a FileMaker backup. BUT we would have to close the files, make the copy, and then re-open it. It’s more complicated and would kick people out of the files. but it would let us avoid this problem.

Hope that helps

Todd

Thanks Todd. I appreciate your quick response.

I’m actually using additional folders in my backup, but we chose to back up the container files as part of the standard backup. I’m still discussing that option with the client, so this may be a reason to change that option.

I haven’t experimented with this - can a backup created by the Admin API exclude containers when the standard setting includes them?

In this use case, I’d probably be willing to close the file in order to back it up. I’d just have to schedule it for late at night. Of course, a backup with just the db file would be ideal.

I just did some experimenting on the server and it seems like an all-or-nothing proposition. If I want to backup containers, I need that feature enabled. In which case I can’t use your wonderful tool to retrieve files from production. This has become a standard part of our workflow (manually copying files from prod to staging after migrations have been completed, sometimes after tweaks are made on production).

I was very excited about using Otto for this part of our workflow because that step of copying back to staging is always a pain. But (at least for this client) the size of the container folder makes this a non-starter. Your solution of grabbing the file from the live hosted may be the best option. If I could run a scheduled script to move a copy of a backed up fm file (sans containers) to specific location, would OttoDeploy be able to access it? Could I use a build for something like that?

Hi Ken,

I have an idea that we could try. I’ll have to do some experimenting. But currently the only way OttoFMS can do it is the way I described above.

You could back up the alternative containers folder to s3 using OttoFMS

Thanks

Todd

Saw you just had a new release. Thought I’d check in and see if you had any more thoughts about this. A great stopgap would be a what you mentioned earlier - the possibility of closing the file on the server and copying just that.

Thanks
Ken

HI Ken,

I think this is not really an issue. When I went back and looked at this problem, I found that external containers were not ever brought into the file.

So could you go back and confirm that this is really an issue for you.

Todd

The issue is not that the containers are brought into the file. The issue is that a backup is run, including all the containers. If the client has thousands of large containers, they’re all copied. For this client, this takes a huge amount of time and disk space. I don’t know if Otto also tries to copy the containers to the staging server as part of the copy script after the backup is complete. I haven’t gotten that far - the Otto script eventually times out.

Right, ok Got it

This version does not address that issue.

Thanks

Todd

Alright thanks Todd.

I don’t suppose there are any hooks to run a shell script or something, Is there? Then I could copy a file from a previous nightly backup or something.

I was so excited for Otto to address this part of our workflow - the alternative is to manually log into the server using RDC and copy the file back to our production server using a web service or desktop mount or something.

Otherwise, Otto’s working great for our migrations. Thank you for such a great contribution to the community!

Ken

Hi Ken,

Sorry, the only way to avoid making the backup of external containers is the method I already mentioned. Use alternative folders and backup the containers using OttoFMS’s offsite backup feature.

We will look at addressing this in the future.

Thanks

Todd