If a complex search is needed, my code is already structured to loop on name variants once inside the bowels of a profile. So, I would accept a proposal for (or explanation of an existing one) method for specifying the syntax, and I would appreciate a tested Regex expression that parses the text into matches so I can just index into sets of FN,MN,LN,BY to use as variants.

Quoting AiAustria:

Quote:
...
I don't know, if it is usefull to seperate the name into F/M/L, because the CLT does the oposite. It equals First Middle//Last with First/Middle/Last...

Eventually though, somebody has to decide, because the underlying database has three fields plus BY. Now that I have recoded to let the user decide, I might leave it that way and shoot my video and see how you like it.

I think I am basically doing what Goodguy did: first run the CLT tool (you can't beat Ken's speed because his is native, and scrape the web pages for profile IDs. Then Loop on the profile IDs, grabbing the credits from the online database. And just as you say he does, this is where I filter for BY.) The web browser I use is built into the plugin, and I am sure that is IE based. But it runs wherever DVD Profiler and the plugin runs. You can use whatever browser you want for your other work.

Well my Clint Eastwood took about an hour on my machine. 18 minutes for the initial scraping to get the list of profiles, and the rest for scanning the XML for credits. I found over 6,000 due to the fact that Eastwood often "does it all".

Here is an example:

Quote:

  <CLTCredits>
    <CLTCredit type ="cast" Episode="" GroupName="" FirstName="Clint" MiddleName="" LastName="Eastwood" BirthYear="0" CreditType="cast" CreditedAs="" Role="Luther Whitney" Voice="False" Uncredited="False" Puppeteer="False"/>
    <CLTCredit type ="crew" Episode="" GroupName="" FirstName="Clint" MiddleName="" LastName="Eastwood" BirthYear="0" CreditType="crew" CreditedAs="" Role="Director" Voice="False" Uncredited="False" Puppeteer="False"/>
    <CLTCredit type ="crew" Episode="" GroupName="" FirstName="Clint" MiddleName="" LastName="Eastwood" BirthYear="0" CreditType="crew" CreditedAs="" Role="Producer" Voice="False" Uncredited="False" Puppeteer="False"/>
    <CLTCredit type ="crew" Episode="" GroupName="" FirstName="Clint" MiddleName="" LastName="Eastwood" BirthYear="0" CreditType="crew" CreditedAs="" Role="Composer" Voice="False" Uncredited="False" Puppeteer="False"/>
  </CLTCredits>

By the way "type" and "CreditType" are duplicate attributes. I will be removing "type".

I will experiment with speeding things up and also running at least two name variants in parallel.

Timing of CLT Plus here in Europe:

John Wayne: about 14mins 11secs
Tom Cuise: about 18mins 47secs
Zhang Ziyi: about 1min 33secs

Environment:
Asynchronous cable internet 150/15Mbit, not heavily loaded, but I watched your video in parallel ;-)
RTT to the farest responding hop on the way to www.invelos.com:
17  144 ms  139 ms  150 ms  c-73-152-128-139.hsd1.va.comcast.net [73.152.128.139]

Concerning running scraping in parallel: This is not possible due to the session handling of invelos.com. If you start more than one CLT Lookup in one browser (two tabs or windows), the results get mixed up.

But it would be interesting to "batch start" more than one query at once.

e.g. after entering First=Zhang, Last=Ziyi you get multiple lines of search strings prefilled with: "Zhang Ziyi", "Ziyi Zhang". Then the user can decide to remove lines from this search strings and add others before starting the whole batch...

Other examples:
First=James, Middle=R., Last=Alexander -> Prefilled Strings: James R. Alexander, James Alexander, Alexander James R. -> the user removes the "Alexander James R." and adds "Jim Alexander" to the search list.
First=Robert, Last=Downey, Jr. -> Prefilled Strings: Robert Downey, Jr., Robert Downey, Jr, Robert Downey Jr, ...

Yes, it is well established that Goodguy used only scraping. In fact, the plugin API didn't allow online access back when he did that code (at least, I didn't think so. What does his downloaded XML consist of?).  And of course I know that the data is available by clicking on the profile links. (Edited)

Two issues: First, I thought people wanted other data. If this is not the case, then what am I doing? Second, so far nobody has been able to duplicate Goodguy's clever scraping methods. I know that I can't, but now that I can access the database, why scrape if you are in a plugin? Now if somebody wants to attempt to scrape the credits, I would be happy to donate my scraping of the profile ID list to the cause. Then, "all" you would need to do is write a program to click on the links, wait for the info to download to browser, and then scrape it. Goodguy was able to do it amazingly fast. I don't have the time or willpower to even try to duplicate that, and furthermore, those methods no longer work in today's browsers (if I am wrong on that, I would LOVE to know about it!!!)

I was not aware of the problem with CLT session handling. I will explore it. When running in a plugin, there would be multiple browser instances, not multiple tabs. Maybe that will allow it to work.

Quoting AiAustria:

Quote:
Concerning the number of profiles in the video: Since each page has 25 entries, it is easy to verify, that Zhing Ziyi has 313 entries in the Invelos CLT (12 full pages à 25 entries plus 13 entries on the 13th page - hopefully nobody around is believing in bad luck )

Maybe an issue with localities?

Yep, I am hoping to get some code settling down enough to release, and along with deciding whether it is worth continuing, we can sort out bugs, refinements, etc.

Quoting mediadogg:

Quote:
Looks like even multiple browsers suffer from the issue...

Thougt so either, but couldn't reproduce it lately...

Quote:
This will still save a bunch of time because there will likely be a lot of duplicate profiles if the variants are for the same person, so the credits scraping for the combined list should go a lot faster than the total of the individual variants' search.

I don't know if this saves that much time. Since the data base is not normalized, each profile lists it's own variant of the name. Potential savings could only result out of profiles, where two different name variants of the searched person are listed for different roles and/or in both cast/crew sections... There are persons using different names for different unions or other personal reasons, but I don't know if their number is high enough to take care of them....