Searches freezing

Question

There are a number of other posts that mention CPU, but I’m not sure whether my problem directly relates, so I started a new topic. Apologies if I should have replied to an earlier one.
I always found PT8 a bit slow, but I have lots of resources scrolling together, so figured that was just too bad. I was hoping PT9 would be faster, but it is just the same. I usually have Chrome open, but even when I don’t it doesn’t seem to speed it up. Some searches are really fast even if they have lots of results (e.g. “reckon” found 275 results in less than a second), but some are really slow, and actually never give any results (e.g. “. (see” went for several minutes before I had to use task manager to close PT, several times). When this happens the CPU seems to be nearly 100%.

image666×593 27.1 KB

I am using windows 10 (1903) and have 16GB RAM. The processor, if it means anything to anyone, is:

Any ideas on how to speed up PT generally or get all searches to work?

Answer 1

Please report these types of issues via Help > Give feedback. No search should take longer than a few seconds on a decent computer.
This type of problem generally comes from the fact that we create a regular expression internally to do the search, and have had a few cases where the generated regular expression creates an infinite loop with a negative look-behind (known as catastrophic backtracking and will never finish). It looks like you have found another case of this problem.

Comments

Reported

---

You linked to documentation from the creator of RegexBuddy. I use this all the time to help me construct good regular expressions, but I know that there are different flavors of RegEx which have different syntax. RegexBuddy allows one to switch between the appropriate flavors, but I’m not always sure of the right one to select.

Can anyone tell me the correct “RegexBuddy” choices for the regex flavors used in the following places:

• Paratext 9 Find/Replace
• Paratext 9 RegexPal
• Changes.txt, FinalChanges.txt, PrintDraftChanges.txt, DBLchanges.txt, etc
• SIL Encoding Converters
• Grep Styles in InDesign

Answer 2

Well, you certainly found the best search expression for the developers to test with. I just searched for
. (see
in a full Bible project, and started timing it. I gave up after 4 1/2 minutes and cancelled the search. I have a 10th generation i7 with 6 cores and 16 Gig Ram, so this will cripple anybody’s computer.

I just tried 9.1 and it does no better.

Comments

If you untick the “Ignore whitespace differences” box (under More>>) does that make a difference?

This is not really a solution, since you might very well want to match one or more spaces, but might give a clue to what is causing the slowdown.

---

12 seconds. It seems like we have a problem with the “Ignore whitespace differences” option.

Answer 3

I can only answer for PT9 Find/Replace, RegexPal, PrintDraftChanges.txt, and DBLchanges.txt. Those all use .Net’s built-in regular expression engine: C# (.NET 2.0–4.8 & .NET Core 1.0–3.0).

Comments

Hidden in this innocuous text is something very important. Are you saying that Paratext 9 RegExPal now uses the same regular expression flavor as Paratext Find/Replace? This is huge and very welcome though I will have to revise my library of regex’s. Totally worth it.

---

Shegnada:

Are you saying that Paratext 9 RegExPal now uses the same regular expression flavor as Paratext Find/Replace?

I think so. It’s a little complicated.

In Paratext 7.x, RegExPal used Python for doing regular expression searches. To simplify the code in Paratext 8, we changed to using IronPython which is an implementation of Python using .Net. Looking at the source code for IronPython, it seems that it uses the .Net RE engine to implement the Python regular expressions.

However, since there are small differences between normal Python and .Net’s implementations of regular expressions, I’m guessing that IronPython must do some small tweaks to the expressions to get them compatible with .Net’s implementation. I’m not 100% sure if it does do some processing of the RE to make it work or whether those changes might make IronPython slightly different from regular .Net or not.

---

I would note that in the Find, the newlines are ignored so you can’t search for something like \\s .* to find section heads. Even if you include the (?-s) the new lines are ignored.

---

anon848905, the restriction you note in Find dialog is problematic. In recent months many of the expressions I share with users no longer work (if they are in pt9) because of this “ignoring newline” reality. I often use “up until” via negative classes… the most common ones being

• until a new line: [^\r]+
• until the next marker (on the same line) [^\r\\]+

These expressions (or technique) for limiting the search to a context no longer work.

I would mention again that this is problematic for giving another user who does not want to venture into RegexPal, or a user who needs to fix a lot of problems one by one in Paratext.

---

One thing you can do to share a complex RegEx Find with other users is to export the RegExPal list, massage it a little with a text editor, and send it to them to open in the List Window.