We're updating the issue view to help you get more done. 

Delimiter workaround in data importer

Description

This may sound convoluted but bear with me. In PastPerfect XML, there are three tags - /subjects, /sterms, and /people - that can contain multiple values. The values (names or terms) are not delimited by any special character or markup. Rather, they simply are separated by appearing on unique lines in the XML file. Like this:

<people>John Doe
Jane Smith
Bill Jones
</people>

For the past few years of running these imports, I've employed a successful workaround where I've put an actual line break in the "delimiter" option in the refiner parameters. Like this:

{
"delimiter":"
"
}

This is invalid json, but it's actually always worked anyway. It was working as recently as December 21 2018 when I was testing new CTCO imports.

However, I've resumed testing this week, and now the load-import-mapping throws and error because of the invalid json.

Would it be possible to soften the load-import-mapping policy to load a mapping regardless of invalid JSON? I know that this is pretty good safeguard generally, but in this case it blocks my workaround.

For context, I've attached the mapping. If you try to load it now, it will throw the invalid json error. However, for evidence of it working as of December 21st, you can view the contents of the following list:
http://ctcotest.whirl-i-gig.com/index.php/administrate/setup/list_editor/ListEditor/Edit/list_id/56

administrator/ctco2018

Environment

None

Activity

Show:
User known
January 4, 2019, 11:31 PM

I upgraded Tumbler to PHP 7.2 over the holiday, which is probably why the JSON parsing behavior changed. Rather than using an actual return try using \n (or \r) and see how that works. Like this:

{
"delimiter":"\n"
}

"\n" is a new-line character. "\r" is a return character. One or the other should work... if they don't we can at least figure out a way to support them. There's no way to loosen the JSON parsing behavior. That invalid JSON was working because of a quirk in a specific version of PHP; there's no option to throw a switch on.

Jonathan Byerley
January 7, 2019, 2:38 PM

Hooray! "\n" worked just fine.

Fixed

Assignee

User known

Reporter

Jonathan Byerley

Labels

Priority

Minor
Configure