Suggestions for handling suspected issues reported by DMT

OttoFMS
, ,
1

It is fantastic how close OttoFMS and OttoDeploy get us to fully automated deployments.

One of the manual pieces that remains is reviewing (and taking action on) the DMT log.

The “detailed log” of a deployment shows the DMT summary, along with the relevant portions from the DMT log. For example:

== Mapping fields in source table “Test” to target table “Test” ==
Matching source field “NameMismatch” and target field “TextField8” by internal id
No compatible target found for source field “NoMatchingFieldInDEV”
No compatible source found for target field “NoMatchingFieldInPROD”
– Record mode migration for source table “Test” –
– Updated the following stored calculations –
Target field “CalcFieldWithEvaluationError_c” and its dependents
Failed while evaluating calculation for target field “CalcFieldWithEvaluationError_c” 1 times
== Summary ==
Accounts migrated: 4
Accounts changed: 0
Custom value lists migrated: 0
Font entries added: 0
Tables migrated: 1
Tables not migrated: 0
Fields migrated: 14
Fields not migrated: 1
Fields triggering recalculations: 1
Fields with evaluation errors: 1
Fields with fewer repetitions: 0
Serial numbers updated: 0

I love how only the relevant pieces are shown.

There are a couple of situations where a human needs to review the log and investigate the cause. For example: if a field wasn’t matched by both name and id, or if a field or table wasn’t migrated, or if there were evaluation errors.

A lot of the time, these are false positives, but sometimes it’s because something was overlooked during development or when preparing for the deployment.

For example, as part of a hotfix, someone added a field in PROD but neglected to do the same in DEV. Meanwhile, a different field was added in DEV in the same table. Those fields have the same id, but we don’t want those fields to be matched during migration.

I’m not sure what the best way to handle these situations would be. Here are some ideas:

If the DMT summary indicates possible errors:

  • Could we have OttoFMS notify us via email and/or text?
  • Could we have a checkbox when preparing a deployment to automatically roll back?
2

Thanks for the ideas. I get the issue you are trying to address. But how do you really know it’s an error. For example I don’t think these

Matching source field “NameMismatch” and target field “TextField8” by internal id
No compatible target found for source field “NoMatchingFieldInDEV”
No compatible source found for target field “NoMatchingFieldInPROD”

are errors.

They are just normal things that could happen depending on the types of changes that you made to the system.

even Fields not migrated: 1 isn’t an error. It just means you added a new field.

I am not sure how you would decide which of these should revert the migrations.

If there was something in the log that clearly indicated that the migration failed we could easily throw an error, and revert the whole thing. But what would that be?

Am I missing something about your idea?

Todd

3

hmmm. thinking some more.

maybe we could do a “strict” migration, that would abort and revert if any potential errors were found.

Then you could review them, and decide to turn off “strict” and run it again. If they were all ok. but I am not sure you’d want to do that on a production server.

Todd

1 Like
4

I’ve found the handling of these suspected DMT errors to be tricky. A lot of times, they are false positives and are safe to ignore. But they can also point to real issues.

I think “No compatible target found for source field” messages are probably always safe to ignore. These will happen if you added a field in DEV but not in PROD.

I think “Tables not migrated” is also safe to ignore. An example of when a table will not be migrated is if it was deleted in DEV.

But I’m always nervous about making assumptions like this. What if there is a scenario I neglected to consider where it’s not safe to ignore?

For these other situations, I don’t know how deciding if something is a real issue or a false positive could be done in an automated way, which means it has to be done by a human.

Fields not migrated

“No compatible target found for source field” messages require investigation. These messages will cause the “Fields not migrated” count to increment.

Some reasons why you might see this message:

The following are safe to ignore:

  • A field is deleted in DEV.
  • A field is converted to an incompatible target, for example by turning a field into an unstored calc.

The following is NOT safe to ignore; i.e. requires manual resolution:

  • You add a field in PROD (as part of a hotfix) but forgot to add it in DEV.

Fields with evaluation errors

Example:

 -- Updated the following stored calculations --
  Target field "MyCalcField_c" and its dependents
  **Failed while evaluating calculation for target field "MyCalcField_c" 30 times**

I don’t know how or why this sort of thing happens, but I do see it occasionally. The way I’ve fixed these is to force the calc field to re-evaluate by making a trivial change to the calculation that doesn’t actually change the calculation result (like adding an empty string to a text calculation).

My guess is most developers don’t bother paying close attention to the DMT summary, in which case the resulting issues persist in the solution until caught later. The cause and effect of such issues quite possibly never gets established then, leading the developer to assume the strategy of not reviewing the DMT logs is a good one. But latent bugs like that where there is no obvious root cause tend to be the most annoying ones to deal with, so I like doing all I can to prevent them.

Having a “strict” mode could be useful. I wonder how much I’d use it though. I’d prefer to try to resolve whatever issue exists instead of having to re-do the whole deployment.

Getting a notification about potential issues would be helpful. Here’s the use case:

  • The users are in the file from early morning until late night, so deployments have to happen overnight.
  • Because deployments happen overnight, I’m not awake to investigate the deployment log for issues.
  • If investigation is warranted, Otto sends a notification. I’d want to configure it somehow so it wakes me up early, before users get into the system.
  • If no issues, I remain blissfully asleep.
1 Like
5

Any more thoughts on this?

6

Hello,

Sorry we haven’t gone any farther yet. It is still an open question.

Todd