Thank you so much for implementing this! You have totally made my day.

So that's a no, then? :/

Link: https://www.sylestia.com/forums/?thread=63707&page=6#52 Author: Moonsugar Time Posted: 7/16/2018 at 7:53 AM So that's a no, then? :/

Can you tell me how you were using the search to go further than 1000 pages in the past and maybe I can better figure out available options.

Well, there are lots of ways to return more than 1000 pages, for instance searching for all of your own pets, or any other user if you are so inclined. Or if you want to see all tagged/themed of one species. Or if you want to see all tagged/themed pets that are for sale of every species. I'm sure there will be more examples now that we have a way to search for purebred offspring as well. :)

Link: https://www.sylestia.com/forums/?thread=63707&page=6#54 Author: Moonsugar Time Posted: 7/16/2018 at 10:49 AM Well, there are lots of ways to return more than 1000 pages, for instance searching for all of your own pets, or any other user if you are so inclined. Or if you want to see all tagged/themed of one species. Or if you want to see all tagged/themed pets that are for sale of every species. I'm sure there will be more examples now that we have a way to search for purebred offspring as well. :)

Okay. But are you not able to narrow the search any further? Like what would be your goal when searching for said things.

Like, let's say you wanted to search for all Themed pets for sale atm. Surely you'd be able to narrow by species or by price, etc. No?

As a quick example. If you search for literally just all Themed Pets for sale, you get over 10,000 results atm. For the first pages of results, the response time is about ~0.11 seconds on average. However, if you skip to the middle (page 500), it's now 4x slower and is taking ~0.42 seconds. By page 990+, it's taking even longer and now taking ~0.68 seconds.

And then some searches by a default, especially if you sort by ID Ascending, take literally 5+ seconds to build the results. So then also add in offsets.

I picked a max of 5000 as a somewhat arbitrary amount based on the hours of testing that I did that I figured would be pretty safe. While there may definitely be room to go up to 7500, 10000, or even more - it definitely can't be unlimited all the way to the max. There are a lot of ways to narrow results depending on what you're looking for, so if you're going through more than 5000 results and literally sitting there going through all 5000+ for whatever the reason may be, you can probably throw in some parameters to make the list considerably less than 5000 and just go from there - like doing 1 species at a time, narrowing prices, etc.

Thanks Krin. I saw no point to purebred breeding or buying purebred pets that had no real difference from any other bred pets. This makes it worthwhile and the high prices for such pets alot more fair. So once again thank you for this solution and for all the other hard work you do for us. You guys are awesome.

Yes, it is possible to narrow the search down in many ways, which is often how it's used. But sometimes I want an intentionally wider search, and I'm not asking for infinite/unlimited search results. Only asking to not make it more limited than it was before.

I can search for each species separately, or each specific theme separately, but that takes a lot longer to set up and if I have to close my browser, all that work is lost every time. Also fests usually have more than one theme (zone theme and "crafted" theme), so that's a lot of fiddling around with the search per species too, because I can't usually memorize all themes in my head. So I find it easier to see all new themes regardless of species.

Also, even at the second page of results from just doing a quick blanket search for all themed and purebred Nixis as an example (just short of 35k pets), it already takes 1.5+ seconds to load one page. Page 2, 3, 4, and 500, 501, 502, etc had exactly the same loading time, for me. Interestingly, searching for every Nixi in the game, which is a larger number of pets (234k), takes 1.2+ seconds to load per page (whether it's the second or five-hundredth page). This is in my browser (Chrome btw), maybe it will differ on yours.

Doing the same test for Draeyl, with quite a bit smaller population of pets, it takes 0.3-0.4 seconds to load each page (again, this is no long testing/parsing, just a quick test). All testing was done with descending order.

One thing of note is that every so often, it will randomly take a lot shorter time (0.0xx seconds, so in the tens of milliseconds) to load the page, I have no idea what would cause it to sometimes load "instantly" and other times take longer, but at least on my end it seems like it is taking longer to load pages now regardless of the 1000-page limit. And it would seem more so the more results you get (with the exception of themed+purebred taking longer, if my indications with the Nixis were correct).

Now I know I could search at least 2k+ pages before the change, and it never took a whole second to load a page, it actually used to feel instant (or not very slow), and I'm not sure if the new search just makes it that way somehow. Do you see a difference between the limit being 5k and 10k? Although you said you could tell a difference between page 2 and page 500, which I don't, so I don't know why that is different for us. But I didn't sit through 500 pages to test that, I did indeed skip to page 500.

A bit confused about my own testing to be honest, but for right now it seems to be the case.

Link: https://www.sylestia.com/forums/?thread=63707&page=6#57 Author: Moonsugar Time Posted: 7/16/2018 at 1:09 PM Yes, it is possible to narrow the search down in many ways, which is often how it's used. But sometimes I want an intentionally wider search, and I'm not asking for infinite/unlimited search results. Only asking to not make it more limited than it was before. I can search for each species separately, or each specific theme separately, but that takes a lot longer to set up and if I have to close my browser, all that work is lost every time. Also fests usually have more than one theme (zone theme and "crafted" theme), so that's a lot of fiddling around with the search per species too, because I can't usually memorize all themes in my head. So I find it easier to see all new themes regardless of species. Also, even at the second page of results from just doing a quick blanket search for all themed and purebred Nixis as an example (just short of 35k pets), it already takes 1.5+ seconds to load one page. Page 2, 3, 4, and 500, 501, 502, etc had exactly the same loading time, for me. Interestingly, searching for every Nixi in the game, which is a larger number of pets (234k), takes 1.2+ seconds to load per page (whether it's the second or five-hundredth page). This is in my browser (Chrome btw), maybe it will differ on yours. Doing the same test for Draeyl, with quite a bit smaller population of pets, it takes 0.3-0.4 seconds to load each page (again, this is no long testing/parsing, just a quick test). All testing was done with descending order. One thing of note is that every so often, it will randomly take a lot shorter time (0.0xx seconds, so in the tens of milliseconds) to load the page, I have no idea what would cause it to sometimes load "instantly" and other times take longer, but at least on my end it seems like it is taking longer to load pages now regardless of the 1000-page limit. And it would seem more so the more results you get (with the exception of themed+purebred taking longer, if my indications with the Nixis were correct). Now I know I could search at least 2k+ pages before the change, and it never took a whole second to load a page, it actually used to feel instant (or not very slow), and I'm not sure if the new search just makes it that way somehow. Do you see a difference between the limit being 5k and 10k? Although you said you could tell a difference between page 2 and page 500, which I don't, so I don't know why that is different for us. But I didn't sit through 500 pages to test that, I did indeed skip to page 500. A bit confused about my own testing to be honest, but for right now it seems to be the case.

Depends on the search - which tables need to be accessed, which columns are being searched on, how the data's being sorted, how many total number of results, etc.

And unfortunately, adding another table into the mix (themed for purebreds) hasn't made many of the searches any faster. Additionally, removing the 20k limit on results has made some searches now take a long time.

As for the rapid response sometimes, it's if you search quick enough in succession that the db still has the previous query cached. However, with how big the tables are and if your query is resulting in a lot of results, the db will quickly purge such query from the cache as it will deem it inefficient to store such a large cached query.


I'm sure I will still play with it and make tweaks and adjustments to it. But I am severely limited just because of how everything is set up. And it leads back to the whole reason for wanting to revamp the entire Site. I can't make those types of changes b/c of how all of the old things are coded. If I were to set up the pet tables to be efficient like they should be, it'd break every other feature on the entire website basically. To make such a change, I'd have to adjust every single other feature which would be an incredibly meticulous and time consuming and mind numbing task. It'd be far easier just to rebuild everything from the ground up from scratch - which is the eventual plan.

this is amazing krin! I'm sure you worked hard on it and thanks so much for this ^^