Shady trick to get encoding converter to skip certain text

Question

I’ve been beating my head against this problem, and just found a solution, so am sharing here in case anyone else finds themselves trying to keep certain text from getting auto-transliterated. This solution is both counter-intuitive and shady.

In a nutshell:

If you have non-vernacular text that should not be auto-transliterated by an encoding converter but rather be published in its original encoding, wrap it in a custom character style, and give that custom style the nonpublishable property.

Yes, the nonpublishable property rather than nonvernacular, as would have seemed fitting. The nonvernacular property does not actually affect what gets transliterated, and the nonpublishable property does not currently affect what gets published, at least not by Publishing Assistant.

Detailed background:

I’m about to typeset an auto-transliterated project. The project contains certain words in the national language. These can be found in glossary entries, in introductory material, and in footnotes, and are tagged within \znp …\znp* markers, which I’d defined in custom.sty as a nonvernacular, publishable character style, because that’s what it was.

In the primary/source project, the vernacular text and the national language text are both in the same writing system. In the auto-transliterated project, the vernacular text should be converted into a traditional script used only for this language, but the text in the national language should come through as it was in its original script. (The traditional script cannot represent the retroflex sounds found in the national language, and it would be weird to see the national language in the vernacular language’s traditional script in any case.)

But marking such text as nonvernacular had no effect on keeping it from being transliterated.

Nor could the TECkit map be told to ignore text enclosed in \znp …\znp* markers, because the markers themselves are not passed into the map for processing. (Presumably they used to, because the Paratext Help documentation still says, “Be sure the TECkit.map file you use preserves the USFM markers so they will not be converted.”)

The tricky solution was to change the publishable property to nonpublishable. This did not prevent Publishing Assistant from publishing it.

I don’t know whether that will mess things up with other output paths (SAB / Print Draft / PtxPrint / YouVersion via DBL), whether currently or in the future they will quietly drop unpublishable text, so that is definitely a concern with this workaround.

Other than nonpublishable exempting text from auto-transliteration, it’s not clear to me what other effects either of these properties might have. Is there documentation somewhere that explains this?

Answer 1

PTXprint (or rather, the ptx2pdf TeX macros that it uses) will treat anything marked nonpublishable as an almost-comment. The python program may also apply its own modifications; I don’t know.

Nonpublishable paragraph and character styles will not produce output, indeed this is one way to suppress verse numbers, etc.
I said almost comment, because it doesn’t treat anything as a true comment: everything still gets read and interpreted, just at the end of the paragraph/character style the material is discarded.
Thus you can’t put \invalid USFM in there, and ranged milestones that start in nonpublishable text and don’t end will still be in force afterwards.

However, given the sequence of stylesheet-loading with PTXprint, you can set something nonpublishable in custom.sty and then override that setting in a later stylesheet.

Comments

Ah, that’s good to know. Thank you!

Answer 2

This is indeed counter-intuitive and it did not work this way in the past. So I believe it must be a bug. In past projects I have quite successfully used \tl…\tl* (Transliterated Word) to block conversion which has the characteristics of publishable nonvernacular. I also used these characteristics for custom paragraph styles in the front and back matter via the frtback.sty stylesheet. And nonpublishable should NOT be coming through Publishing Assistant to InDesign.

Hopefully we can get this behavior corrected so you should be flexible with your markup till then…

Blessings,

Comments

Thanks, Shegnada. The original markup was indeed \tl ...\tl*, which may reflect that at some point in the past, that worked in this project to prevent auto-transliteration. (The NT was published several years ago, and now we’re doing the full Bible.) In that case, this bug is a regression, something that broke when something else was being addressed. I’ve reported it as PTXS-31555. Once it’s fixed, it will simply be a matter of updating the definition of \znp that I have in custom.sty and frtbak.sty to replace nonpublishable with publishable.

But now I wonder if the functionality was intentionally changed based on someone reasoning that \tl text should be fed through the converter. This project has since been using the \tl marker in the OT to mark words transliterated from the source languages, such as purim and mene, tekel, parsin. The translation team’s intent is that these words will be processed by the converter to be in the same script as the project, but be italicized in whichever script that is. So I guess when this bug is fixed, we’ll also need to remember to redefine \tl as vernacular in this project, to ensure it does get processed by the converter. Does that sound like the way to do it?

I’d still really love to find documentation on exactly which effects the publishable/nonpublishable and vernacular/nonvernacular properties are intended to have. So far I only know that nonpublishable should result in the text avoiding encoding conversion and also avoiding being visible in publications. I believe that nonvernacular is intended to avoid encoding conversion (though that’s broken), and I’m unaware of any other intended effect. I wonder if either of them have a bearing on spelling wordlist, character inventory, etc?

---

I think that as long as you use a custom stylesheet, e.g., custom.sty in your case and document with hashtags what you are doing and why AND add a Paratext note at the first \tl location the behavior which you are looking for, you should be ok on your PubAssist/InDesign path.

But the replies you got to your original post would indicate that if you have other outputs in mind, you should test those also and perhaps a custom marker would serve you better. I don’t really know.

You should take into account that you have two, or perhaps as I normally have, three, projects involved. The original project, the automatically converted project (uneditable), and (I recommend) a third project used for publishing which gets its text from importing from the converted project. Each of these projects can have different custom stylesheets and while it is important that the first project have the style characteristics that will make the converter work properly, the stylesheets of the project you publish from can be different from these and they are the only ones impacting the published output. Thus you have some flexibility, especially if you use that third project. I can talk with you more offlist if you like about the other advantages of a third project, but I would consider the second and third projects to be output merely and documentation, etc., should always be found in the original project.

Blessings,