We're updating the issue view to help you get more done. 

New log-level that tracks dontCreate setting and errors in data import

Description

Create a log-level that notes every record where the dontCreate setting prevents a relationship from being matched/made. Include this with errors (no informational, warning or debug messages).

Environment

None

Activity

Show:
Seth
December 21, 2019, 8:40 PM

Implemented in develop branch. Added additional logging of dontCreate + new app.conf setting (“log_import_dont_create_events_as_errors”) to force logging of dont_create events as errors.

Julia Weist
December 23, 2019, 3:57 PM

Could we add the idno of the imported record on which the dontCreate match was attempted? For example, instead of this:

Not adding "Canada, General" (767163) as ca_places because dontCreate option is set

We’d have this:

Not adding "Canada, General" (767163) as ca_places for 8000-1-5 because dontCreate option is set

 

Seth
December 23, 2019, 4:21 PM

I've implemented a version of this in develop. The 8000-1-5 is up front in brackets, so it'll be more consistent across all error messages.

Julia Weist
December 23, 2019, 4:48 PM

Thanks! Here’s what I’m getting now:

2019-12-23 16:34:34 - ERROR --> Not adding "Canada, General" (767160) as ca_places because dontCreate option is set
2019-12-23 16:34:38 - ERROR --> [1987-51-60-x] Not adding "Canada" (767161) as ca_places because dontCreate option is set
2019-12-23 16:34:38 - ERROR --> Not adding "Canada, General" (767163) as ca_places because dontCreate option is set
2019-12-23 16:34:39 - ERROR --> [1987-51-61-x] Not adding "Canada" (767164) as ca_places because dontCreate option is set

The mapping uses “{"useParentAsSubject": 1}” across five columns. In these examples the first two child-most columns are empty and then it finds “Canada” which would be a valid match. However, the next-level up parent “Canada, General” doesn’t exist and dontCreate kicks in.

So I think we just need to account for this mult-level matching behavior in the logging. We want to keep the error for “Canada, General” because that’s where the data doesn’t match, but either it simply gets the [1987-51-60-x] as well, or it could be on the same error line as the child.

Fixed

Assignee

Seth

Reporter

Julia Weist

Labels

Components

Fix versions

Affects versions

Due date

2020/01/07

Priority

Major
Configure