Performance issues with large collections

Discussion in 'Feedback and Suggestions' started by Oberon, Dec 21, 2013.

  1. Oberon

    Oberon Hydra

    The size of my collection allows me to test the card hunter client’s ability to handle large numbers of items. Based on what I’ve seen I’ll share a few concerns.

    I do what I consider to be an average level of maintenance on my non-treasure items. Once or twice a week I sell these items down to the minimum number needed for a hypothetical party. I keep the maximum number of items you would potentially need; 12 of each arcane item, 9 of each weapon, 6 of each staff, 3 of most items. I’ve been imitating Scrooge McDuck with my treasure collection, keeping all items until today.

    Given the set design, a player can reasonably grow their collection well into the 1000’s even if they are continually selling items. For example, there are 46 marital skill items currently in the game, if a player keeps 3 of each they end up with upwards of 138 items. Other types of items can be worse, just keeping to rare, uncommon and common arcane items, collecting 12 of each results in 111 items X 12 for a total of 1332 items. I estimate my non-treasure item collection to be around 4000+ items at the moment.

    My treasure collection had grown to what I estimate to be an additional 4000 items. And for the last month or two performance has not been great. Loading into adventures or trying to use the collection interface has been less than optimal. I’ve gotten used to dealing with lag times of multiple seconds, though I blame myself due to the size of my collection. That collection is being loaded very regularly.

    treasure.png

    Today I cleared up my treasure collection, clearing out around 2000+ items from my collection. This has made a noticeable improvement to the performance I’m seeing. Which matches to my experiences selling 1000s of items off in the beta, but it also concerns me.

    Right now, I expect there are relatively few players with that many items, but I expect it will be more common when new content is released. With a theoretical level limit of 50, we could see item counts triple from what we are currently dealing with. With the potential release of additional classes, other substantial increases in the item count would occur. I’m curious to the performance of the client when item collections could contain 10,000+ items.
     
  2. Phaselock

    Phaselock Bugblatter

    Lusus121, spacedust and Sir Veza like this.
  3. Sir Veza

    Sir Veza Farming Deity

    That sounds ideal to me. Better organization, better visibility, better control, and less strain on the computer.
     
  4. Jarmo

    Jarmo Snow Griffin

    I see the same slowdown on my "Jarmo" test server account. I opened 2,000 Epic chests and a few hundred Magnificent chests on it. I've had to sell all excess items as in the opening post and the performance is still too slow by far. If you have a fullish collection even with the current amount of items, the game can't handle it. Something needs to be done. The proposed "show only one copy with count" solution sounds very good!
     
  5. Jarmo

    Jarmo Snow Griffin

    Farbs in the thread Phaselock linked:
    Please start optimising!

    Showing only one copy of an item has other benefits, though, so please start to use it even if you could get enough performance with the current item display. Less UI work for the player, less scrolling, less need for filtering, easier to get a comprehensive picture of your collection, more informative inventory screenshots etc.

    As item storage space is not a gameplay issue in Card Hunter (no inventory Tetris or premium Bags of Holding or 20-slot belts) it makes no gameplay sense to display every copy, either. It just makes the item management interface simpler with no need to enter the number of copies to be sold. It can still be kept simple with the proposed model, e.g. just make drag- or ctrl-click-selling always sell a single copy.

    I still advocate the "sell all excess items" button that is regularly proposed. That is the single best thing that would make playing the SP game more pleasant at the moment, followed by the performance enhancement.
     
    Flaxative, spacedust and Sir Veza like this.
  6. Sir Veza

    Sir Veza Farming Deity

    +1
     
  7. spacedust

    spacedust Goblin Champion

    This would be on my Christmas wishlist too. For a frequent farmer, this issue really detracts from the game.
     
    Flaxative likes this.
  8. Flaxative

    Flaxative Party Leader

  9. Farbs

    Farbs Blue Manchu Staff Member

    I've added a TODO to look at this, and will tape a reminder to my monitor so I won't forget once I'm back "at work". If you toggle on verbose mode in the console (type "verbose") then enter the deckbuilder you'll see the first thing I need to do. Currently your inventory is sent with one chunk of data per item. Rewriting this format as item ID and count should improve load times for large collections.

    Of course, I could be wrong about what's slowing things down here. If the problem turns out to be display object instantiation then I'll have to have a bit more of a think about it. We'll see.

    Now, where are my postit notes...
     
    spacedust, Jarmo, Oberon and 3 others like this.
  10. Jarmo

    Jarmo Snow Griffin

    Thanks for the console hint, Farbs! With its help I was able to determine the total number of items on my accounts. My live account currently has 4,454 total items and the test account I mention above in post #4 has exactly 6,000 items (a round number just by amazing coincidence!)
     
  11. neoncat

    neoncat Feline Outline

    Just wanted to bump this again. I froze the game by dumping ~500 items into The Armory after a couple farming runs. My collection only has 2260 items (after the dump), but at the end I was seeing 4-5 second mouse input lags up until it froze.
     
    Flaxative likes this.
  12. Forestal

    Forestal Mushroom Warrior

    Yes, it would be great to have a feature/interface where the number (of copies/duplicates) of an item you own is simply displayed/indicated by an number/value floating above or attached to the icon of the item-- it's cool/crazy to see every single icon of every single item lined up like that, but it's a mess to go through them...

    Probably needs an overhaul of the interface-- User-Interface 2.0?
     
  13. Flaxative

    Flaxative Party Leader

    I just sold 1400 gp worth of commons & uncommons...

    [​IMG]

    Not noticing performance improvements if any, though maybe that's because I'm still holding onto too many copies of items?

    (To be clear, where performance is mediocre for me is when loading Keep, stores, and modules—any new view that calls on my inventory. The loading icon stalls for a good 15+ seconds after spinning and before letting me into the Keep/store/module. Everything else about the game is smooth for me.)
     
    LeisureSuitLoli likes this.
  14. Jarmo

    Jarmo Snow Griffin

    Have you tried checking how many items you have total? It's a bit tricky to do, but I've noticed the more you go over about 4,000 items the more you suffer. 6,000 is too slow to want to keep playing much. The loading times are the killjoy. Once you're in a battle there is no slowdown.
     
  15. Flaxative

    Flaxative Party Leader

    Looks like I've got about 3200 after that cleanup (which involved selling over 1000 items). I'm still getting lag as I said, though it might be slightly less than before—hard to tell.
     
  16. Farbs

    Farbs Blue Manchu Staff Member

    I've just submitted two improvements to this, which should appear in the next build:
    1. Load times should be shorter - my test case went from 20s to 6s, which is still long but obv much better.
    2. Scrolling through the collection should be much smoother, most of the time. We were performing a lot of redundant sorts, which I've removed.
    These won't magically fix every issue everyone encounters with large collections, but it should help a lot. The nature of optimisation is that there's always more you can do, usually for diminishing returns. This seems like a good start though.
     
    Sir Veza, Jarmo, turinturamba and 4 others like this.
  17. Sir Veza

    Sir Veza Farming Deity

    Does this use item ID and count? Just curious.
     
  18. Farbs

    Farbs Blue Manchu Staff Member

    The first part does, yeah. If you use verbose you'll see I just laid down a lazy stripe in an int array, alternating between item id and count. There's still plenty that could be done, eg representing it as lists of item IDs that have a particular count, each led by the # of items in that list. For most of us, where our counts for any particular item are <6, that would nearly halve the data load again. We could also use more efficient data types, since we don't need 32 bits to represent each item ID.

    On a project many years ago, where we needed realtime bumper car physics across a 64kbps network, I proposed a sub-bit system of data compression where a particular bit could impact on two completely independent pieces of data. My proposal was turned down for being, frankly, a bit silly - but it would have worked. In more recent years I've used all sorts of non-multiple-of-eight bit lengths to encode things, largely for fun.

    The point of all this waffle? Optimisation is a rabbit hole. There probably is an end to it somewhere, but it's always further than you think, and there's rarely any point going there.
     
    Sir Veza likes this.
  19. Sir Veza

    Sir Veza Farming Deity

    Been there, got stuck.
     
    Farbs likes this.
  20. RedLineFire

    RedLineFire Kobold

    Thank god for my pseudo-OCD to kick-in and say I need to clean my collection.
     

Share This Page