Invalid or unknown character check should not look at \tl

Question

Hello, many of you might rarely use the \tl, so here is the definition from usfmReference2_4.pdf:

image.png1174×417 112 KB

So it is obvious that “unknown” characters almost must happen inside any \tl Ik dee tell juu!\tl*

And all those false alerts are making the checking-work really bad. I always tell people here: Never ever get used to ignore (faulty) alert-lamps in your car. When you have the next real problem, your brain will just ignore the little red light and your engine might all seize up from lack of oil or whatever.

I looked this up in our custom project style sheet, this marker is still at its default settings:

\Marker tl
\Endmarker tl*
\Name tl...tl* - Special Text - Transliterated Word
\Description For transliterated words
\OccursUnder ip im ipi imi ipq imq ipr iq iq1 iq2 iq3 io io1 io2 io3 io4 ms ms1 ms2 s s1 s2 s3 s4 cd sp d li li1 li2 li3 li4 m mi nb p pc ph phi pi pi1 pi2 pi3 pr pmo pm pmc pmr q q1 q2 q3 q4 qc qr qm qm1 qm2 qm3 cls tr th1 th2 th3 th4 thr1 thr2 thr3 thr4 tc1 tc2 tc3 tc4 tcr1 tcr2 tcr3 tcr4 f fe NEST
\TextType VerseText
\TextProperties publishable nonvernacular
\StyleType Character
\FontSize 12
\Italic

As you can see, it is tagged as nonvernacular. So I wonder why the invalid or unknown character check is even applied at all. What else could we do, to un-alert this check for anything we need to put into marker \tl?

I have not filed this as a bug, still hoping we are just missing a configuration. It was very thoughtful to create this option of having foreign words in PT, but to work properly those need to be treated special during checking.

Answer 1

These are those places where the checks get quite involved. With characters things can get even more complicated. Any character found in the language should be added to the Alphabetic Characters tab of the language settings. This will cause them to be automatically marked as “Valid” characters in the Character Inventory.

All other characters are marked as “Unknown” until someone marks them as either Valid or Invalid. In the case of characters in the \tl…\tl* field, these will show up as unknown because the are characters in the text and the inventory is showing the characters in the text.

Unfortunately there is currently no way to identify characters as only valid in certain circumstances.

There are basically two choices - 1) mark them as valid, but in this case if they show up somewhere where they shouldn’t they will not be identified as questionable, or 2) mark them invalid (or leave them unknown) in which case the will be reported as errors and you can “deny” the error by using the Edit > Deny Error option in the list window.

Comments

anon848905:

Any character found in the language

Yes, that is exactly my question, as anything inside the \tl markers is by definition not in the language. We discovered this problem, when we added some Greek terms for coins into our glossary entries.

anon848905:

2. mark them invalid (or leave them unknown) in which case the will be reported as errors and you can “deny” the error by using the Edit > Deny Error option in the list window.

So @anon848905 this second option is the solution to our need: Since those \tl occasions are not too many and since the “deny-error” is not working globally but specifically for each occurrance of a certain error (I just did some tests, as this is not mentioned in the inbuilt-help), I can manually deny those false-errors in our glossary and still stay alert for such typos in the main text.

The PT help is saying this:

After you run a basic check, you might decide that certain reported “errors” are not errors, but instead are deliberate deviations from what was specified in the inventory, rules, or settings for that check. You can “deny” such “errors” so that they don’t appear in the list the next time you run the check.

And it might be helpful to add to help that this denial, is working on a case-by-case system. So if I find the character “ö” in LUK 17:10 and I deny this error (for whatever reason), then another occurrence of “ö” would still be flagged by the Character Inventory check.

For the Shotcut video-editor we are using the same forum-program. And there they have a feature where you can mark a thread or a question as “solved”. I do not see this option, but maybe one of the forum administrators can do this. Because this user has found a pragmatic working solution. User is happy.

Thank you anon848905!!

---

Tim:

Yes, that is exactly my question, as anything inside the \tl markers is by definition not in the language . We discovered this problem, when we added some Greek terms for coins into our glossary entries.

Just to add one more clarification - or possibly muddy the water. The character inventory is NOT an inventory of the characters in the “language”. The characters in the language should be listed in the Alphabetic Characters of the Language settings. The Character Inventory is a list of ALL characters in the text. Therefore even though a Greek letter might not be a character in the language, it is a language in the text and shows up in the inventory.

---

Yes, thank you. We have properly filled out all the Language settings from the beginning and had another walk-through after the migration to PT8.

In normal work, the character inventory is mainly helping us to find typos and some junk that would sneak into a text through various hiccups. That is why I am reluctant to declare valid certain characters that are not part of the main language.

---

The philosophy I teach in our Scripture Typesetting Workshops is if the character is listed in the inventory but is not valid inside the rest of in the scripture text is,

• mark it as invalid in the Inventory

• close the Inventory

• run the character check

• deny the error indicating you have checked it in that instance and it is valid there.
  That’s the concept. Here are the actual steps I perform to accomplish that.
  What I typically do if there are a lot of other language characters only valid inside the \tl \tl* markup is :

• run the character check.

• In the List Window, click Sort from it’s menu bar

• select sort by message.
  That should group the list of common characters.
  Since you are wanting characters in the \tl markup I use a regular expression that looks for occurrences of that character inside the \tl \tl* markup.

• run find

• enter the following expression in the find box:
  regex:(?<=\\tl[^\\].*?)[kKjJzZ](?=.*?\\tl\*)

• Change the letters [kKjJzZ] inside the square brackets to your list of special characters used only inside the \tl markup—but that should not occur in the rest of the scripture text.
  The list inside the [ ] should look like this:
  [kKjJzZ]
  which is looking for lower and upper case versions of the letters j, k, z.
  

• click on the “Create List” drop down list in the bottom right of the dialog and

• select “Shared With List”.

• Then click on the “Find” button. What you will get is a filtered list for those particular characters.

• Select all the matches in the List Window by

• clicking on the first entry,

• hold the shift key while scrolling to the bottom of the list,

• while still holding shift, click on the last entry,

• With all of the \tl only characters selected

• Click on Edit in the List Window,

• and then click on Deny.
  Then rerun the Character Check and deal with only bad characters in the rest of the scripture text.
  Hope this helps with denying those special characters that should only occur in \tl text.

D anon467281

Admin edit: Attempt to fix formatting problems from e-mail.