[REQUEST] Consider separating the fetching of game details from search.

[xavdid]

Describe what change or improvement would you like

It seems like the search url is the one HLTB keeps changing. While your HTTP parsing code means you only have to worry about the one page / url, it also means that fetching game details starts to fail as soon as search is broken.

Given that they don't often change the details page, It might be worth parsing it directly in the search_from_id methods so it's more stable (even when search by name periodically breaks).

Happy to help out with this (or any of the issues I've been opening; sorry for the deluge)!

[ScrappyCocco]

You can provide a PR with your change so we can check it out and discuss it!

[ScrappyCocco]

Also at the time years ago I did that the search by id is actually searching by name to reuse the same parser, how do you search directly from id in your api?

[xavdid]

how do you search directly from id in your api?

Not my code, but the JS version of this library parsed the HTML on the detail pages (e.g. https://howlongtobeat.com/game/5037), which never changes. Their implementation is here: ckatzorke/howlongtobeat | howlongtobeat.ts#L105-L113. That way, the packages does 2 things:

• turns a name into a HLTB ID via the search method (have to update this a lot, since they're changing it)
• turns a HLTB ID into playtime information (changes infrequently, low maintenance)

As written, you lose both pieces of functionality when the search breaks, since your implementations are coupled together.

[ScrappyCocco]

Parsing the HTML requires maintaining another parser which may break virtually anytime and doesn't seems a viable or better solution to me
That API parse the HTML because it has not been updated in years since when we changed the code to parse the search json, and at the time it was the best solution, this API was parsing the html too at the time if you go search. Now I feel like this solution works better
If you can propose an improvement using the current parser/logic then we can discuss it

[xavdid]

Parsing the HTML requires maintaining another parser which may break virtually anytime and doesn't seems a viable or better solution to me

That's true, but the whole thing already breaks all the time because of changes on the HLTB side. Isn't it better for search to break the same amount, but other functionality to break less?

parse the HTML because it has not been updated in years

the HTML on the game-specific pages also hasn't changed in years. So it's less of "maintaining a second parser" and more of "write a second parser once" (which I'm happy to submit a PR for). It would be the same amount of maintenance, but you'd go from "everything breaks" to "some functionality breaks", which seems like a worthwhile improvement to me. 🤷‍♂️

[ScrappyCocco]

Right now I don't like the idea of having and maintaining a second parser: at the time the HTML parser had many problems and was not very efficient and easy to maintain and write, and yes it required changes. I'm more into having a system that recognize the search call as for example #45 and hopefully other similar systems in the future, if you find a good solution to make a good parser feel free to open a PR for it and we can discuss, but I don't like the overall idea.