[Utility] Collection Analysis

Discussion in 'Card Hunter General Chat' started by neoncat, Feb 1, 2014.

  1. Stexe

    Stexe #2 in Spring PvP Season

    Sounds good, I'll try it out tomorrow or so.
     
    ParodyKnaveBob and Sir Veza like this.
  2. Stexe

    Stexe #2 in Spring PvP Season

    Sir Veza likes this.
  3. Sir Veza

    Sir Veza Farming Deity

    Probably more for @PhoenixTheHunter , but Max Usable shows 96% in Collection Analysis, and 94% on the Collection Badge. I've included a screenshot of each, and the data used to produce them.

    analysis.png Badge.png
     

    Attached Files:

    ParodyKnaveBob and Stexe like this.
  4. I was about to say, that this is a very interesting incident, but I found the (first) problem with just checking the screenshots.

    First of, including promo items while a) it is neither Christmas nor Halloween AND b) you excluded undroppables, does exactly nothing. Promo items are the seasonal duplicates but they are undroppable if we are out of season. So, either you have to include undroppables, too, or wait for the season for them to be included. But that isn't the issue here as both tools should exclude them in your setup (and at the time of writing). If you want to include undroppables in Collection Badge, you have to include both undroppables and promo items in Collection Analysis to again have the same item selection. One tool has fewer options, because for Collection Badge you want to have an overall view of your progress and that badge shouldn't be able to change based on time of creation (only based on options that are reflected in the small icons). While for Collection Analysis you have more options (including the filter rules) to extract the items you actually care for.

    The issue is simply treasures (edit: or so I thought at first, this is still true, but probably not the main reason). Both tools have different defaults: Collection Analysis excludes them by default, Collection Badge includes them, so you have to change this on one of the pages to achieve the same selection. Again, both tools have different use cases, hence the different defaults. (For you, the badge says 94% either way.)

    While checking the numbers again with one of my collection dumps, I discovered another (and the actual) problem that would result in these numbers never being the same. See, the Collection Badge checks for every single item if it should be counted (aka whether you haven't excluded it by your selection) and then if you've collected enough copies to be a full set (at least 1 for normal mode, or 3/6/9/12 for max usable mode). Hence if you have just 2 copies of some item, it isn't counted as full in max usable and therefore is counted as 0. That means the Collection Badge tool only counts each item as "1" if it is full. This is a binary decision: full=1, not full=0, regardless of the mode. That is similar to the "can equip a whole team?" mode in the Collection Analysis. So, to have matching numbers you have to compare not "max usable" (for Badge) with "one copy of each item" (for Analysis) or "max usable" (for Badge) with "can equip a whole team?" (for Analysis). At the time I introduced this "max usable" checkbox, no-one objected to this setting being equivalent to "can equip a whole team?" and this was the easier way of implementing this feature, therefore this is how it works now. ;) Unless a majority vote favors the other (more difficult) way.

    TL;DR: There are items where you have collected some copies of but not enough to have a full set. One tool interprets this as x copies, the other as "no full set", hence the difference in numbers.
     
    Sir Veza and ParodyKnaveBob like this.
  5. Now that I finished my earlier post, I am actually wondering whether the Collection Badge treats seasonal/promo items the way I described it. I mean, it should. The badge should not change over time just because one promo set is available every now and then. The small icons should always be unambiguous. But based on a small inspection, the tool might actually ignore the promo-ness of items altogether and these items could switch from undroppable to droppable (and back) once a year (given the devs start and end the respective game events). So, thanks Sir Veza, you might have unknowingly (and indirectly) reported a bug. :cool:

    @Stexe I haven't had the time to check the tools, but I will do so the next days.
     
    Sir Veza and Stexe like this.
  6. Stexe

    Stexe #2 in Spring PvP Season

    Sounds good.

    I have an old version of the tools up at: http://wwww.Stexe.com/CardHunterOld/ for those looking for the previous hosted ones.
     
  7. So, as far as I can tell, the new hosted files work as expected. I haven't tested every single feature, but each tool at least does something that seems legit. I will do some more checks over the next days.

    I already "fixed" the collection badge and did some more housekeeping with my code. Also, I wrote a simple (Linux) script to update the resource files (either manually or periodically with something like cron). Given that I provide three different archives and packaging them isn't automated, I will wait a bit more and see if you all find other issues with the tools that I can address all in one go.
     
    Sir Veza and Stexe like this.
  8. Quick addition: I think (I'm still quite inexperienced in php) I found a way to merge the fully automated and the semi automated version into one. That would make it easier to package them all for a release as it would reduce the number of artifacts to two. Then, there again would be the version completely independent of php and the version dependent on php that - by invoking the resource-file fetching job once - could be turned into a semi automated version. And turning the semi automated version back into the fully automated version would be done by simply deleting the "cached" resource files. How much simpler could it be.

    PS: Anyone around who is familiar with bash programming for Windows or Mac? I would perhaps be able to convert my Linux script into Windows bat format (but I don't want to, honestly, I'm glad to have left Windows behind at that point) but I don't have any experience with Mac console scripting or capabilities to test the result at all. If you can, dm me and I'll send you a preview of my current resource-update-script for Linux. If no-one is willing to convert the script, I will only package the Linux script in there (as most hosters use some Linux distro after all).
     
    Sir Veza and Stexe like this.
  9. Sir Veza

    Sir Veza Farming Deity

    Thanks for a) explaining how the max usable collection badge works, and b) leaving Windows behind.
     
  10. I mean, explaining the tools is my job, but for this one, you're welcome ;) EDIT: You know that two of your four signature badges don't show up?

    Regarding the update jobs for Mac and Windows: I would still like to have someone verify this, but I have a feeling that the situation is better than I initially thought: Mac and Linux have a common basis, so given that my script isn't making use of fancy commands - at least I think so -, this script might actually work in Mac natively. And for Windows, there is WSL - Windows Subsystem for Linux - basically a integrated Linux VM. Now, WSL isn't trivial to use and if someone manages to convert my script into a bat or powershell script, this would make things easier for Windows users, but if nothing else, the script should be executable under WSL no problem.

    PS: Another reason why I'm not releasing another version right now, is that I still hope that the latest announcement means new additions for the tools.
     
  11. Stexe

    Stexe #2 in Spring PvP Season

    If there are easy to fix issues I'd say go for that now and I can upload a new version. The new Card Hunter additions could be quite a bit away and I'd like to have a stable build until then.

    Was the only problem with the collection badge? Any other issues?
     
  12. Okay, I will factor this in, if we discover an issue that actually is an issue. Up to now, the only thing I found is the collection-badge-promo-item-filter-issue that is not really an issue. So, I will wait for now.

    I attached the update script (also for those considering porting it to or testing it on Windows or Mac). This isn't final yet but it would be awesome if you could test this for once. Unzip it and place it in CardHunter/resource (this folder already exists), you might have to make it executable (
    Code:
    sudo chmod +x update-resources.sh
    ) and execute it and see what it does. Expected behavior is some status messages of it downloading four files into a temporary folder, and then this folder should be deleted again and the four resource files that are already present in the resource folder should not have changed (because there are no new Cards/Items/Adventures/Figure-skins). (you can also dm or mail me if there is something unclear so that this thread isn't spammed too much ;) )

    EDIT: new script version with much more readable output (and also improved logic)!
     

    Attached Files:

    Last edited: Apr 6, 2021
    Stexe likes this.
  13. Sir Veza

    Sir Veza Farming Deity

    I'm having a forum problem. When I try to upload a changed file with a different name, and even from a different folder, the old one still displays. Getting pretty disgusted with it at this point.
     
  14. Whatever you did, now I at least see no more broken image links anymore.

    Given that I work on code that I last touched almost two years ago, I again have to get familiar with all the magic that is going on in every corner. And some of it is quite dark magic. And all of this on a new system, with a different operating system and other software (versions). But I think I can say that the hosted files seem fine so far. I found a few things that I will change to different colors. But that is no issue with the hosting, just that I have a different perspective on some of the utils now. When I first loaded all my sources into my new tool chain, I got some warnings that I will fix properly (these warnings shouldn't break the tools, but if you get aware of a potential issue, why not fix it). I will have to test my collection-badge-promo-filter-fix and the merge of sau and dev version (don't want to rush those things and risk them to break). But today I worked on the update script again and a build script that will with one command create both release archives at once. That should make it impossible for me to miss files (which is possible based on two years without release, so I better test my build script, too). For this all to be accomplished, I guess I will need a week or so, hence, try your best to break the tools now, and you might get fixes for those issues immediately with the upcoming release. :cool:
     
    Sir Veza likes this.
  15. I guess I was quicker than I thought because everything seems done so far, except for the "try if something breaks" tasks. But with the new system, this should be even simpler than before. The resource cache lets me supply arbitrary resources to test specific issues. It's not some seasonal event but I want to see how the tools would behave? Edit the resource file. Some specific item behaves differently? Delete all other items and debug with just the one. Not sure the campaign badge excludes/includes the right adventures? Create test files that should display very specific data and just watch the output.

    I won't do all of them, these are just a few examples of why testing is now really comfortable. And my new build script will refuse to create a release if there are any code issues left. Hence, one more instance checking what I do. ;)

    While working on my build script I was wondering whether I should rename all php-files to html in the test-version. Given that this is now automated, there is no additional effort necessary for each individual build. There is no more php-code in them at that point and some systems refuse to open php-files with a browser. I guess I will do that while testing the new version and waiting for other bug reports from you all. So, are there any more issues???
     
    Sir Veza likes this.
  16. Okay, folks, I'm a bit tired of clicking through all the tools myself, so here you have it: The new version of the utils. Now, it's up to you to find all the remaining bugs. I count on you!

    As said before, we are back to two versions, the dev-version and the test-version. Both versions include a (identical) README, so please read it! If you don't, I might answer with RTFM it you ask something that I explained in it. And everyone should now be able to use the test-version because all the files are from now on html instead of php.

    Regarding changes: There aren't many, well, most of the files changed but not in their function (most of the time). We have a defined background color for tools that previously hadn't one (and probably were white for you, because most browsers use white if the page doesn't specify a color). The collection badge tool handles seasonal reprints as strictly "noLoot". I changed the color for attack cards in quick draw card pool slightly for a better contrast. And obviously, I included the semi-automated-update feature into the dev-version such that the dev-version now can be used as always up-to-date or based on some pre-downloaded resources. Before you ask, how those different "states" work and can be converted into each other is explained in the README. I will not describe this all in more detail because I think I did so already in previous posts here. If there are still questions, feel free to ask.

    For the scripts: There are a few things I want to address here. If you made them executable ( sudo chmod +x <script-name> ) and you execute them by double-clicking in some file browser, everything should work fine. If you execute them via shell, make sure that you are in the same folder as the script. Each script will assume it is executed in the same directory, if not, if might behave oddly and not do what it is supposed to do. There is nothing I can do to 100% ensure that the user calls them correctly, so I just ignored the possibility of abuse completely. It is your system you are affecting by executing a third-party script and I'm not reliable for any damage you do to it (I hate that you have to add this to any license-file or whatever and I haven't even added it to these releases, but here is your warning. Everyone still clicking on every executable they are getting their hands on in times of ransom ware or other malware doesn't deserve better). If you are familiar with Linux shell scripting, feel free to inspect the code, you might even find features that are disabled by default but that I liked to include from the get-go. If you are familiar with scripting on more than one OS, feel free to port these scripts to native Mac or Windows scripting formats and repost them here and I will include them in the next release (if I saw that you haven't completely butchered my script or included malware). If you don't speak code, and you made no mistake with clicking on them, I would be 99% sure, that you can't damage your system. Finally, the scripts are written for pretty much every Linux system and Linux shell (they target "sh" but any other shell, like "bash", "dash", ... should work fine, too). They only use the tip of the core commands, so to speak. Every Linux install should be natively supporting and installing every used command. If not, then you are a Linux user and capable of installing wget on your own, d'oh.
     

    Attached Files:

    Sir Veza and Stexe like this.
  17. Stexe

    Stexe #2 in Spring PvP Season

    I can put the new stuff up (I assume the "dev" version?) in a day or two.
     
    Sir Veza likes this.
  18. Yeah, the dev-version. But I'd say give it a few days. There was nothing inherently broken with the version you are currently hosting and perhaps someone will spot a serious issue with the new version before then.
     
    Sir Veza and Stexe like this.
  19. Stexe

    Stexe #2 in Spring PvP Season

    Will do, hopefully someone can spend the time to test it out. =)
     
    Sir Veza likes this.
  20. ParodyKnaveBob

    ParodyKnaveBob Thaumaturge

    I finally got all this working. On your website. Yay.

    upload_2021-5-29_16-45-58.png upload_2021-5-29_16-46-15.png

    I was rather surprised how high the one-of-each number is. $:^ J
    Excluding undroppables, it's 95%.

    Thanks to the excess checker, I was able to quickly find another 20g or so in accidental commons and uncommons.

    Thanks to the analyzer, I see it wouldn't hurt if I weighted my Loot Fairy Madness toward level 8 (6—10 are all missing something) and ~18 (except that there are only so many high level adventures I want to complete in co-op and/or on repeat, lol. Hey, I already saved the world. Role has been played). That said, I only 4 Epics $:^ ] but 103 Legendaries. $8^ |

    Thanks to @neonko-sensei (that's @neoncat's new name, right?) in the first place, plus Stexe and Phoenix for the current availability and updates!
     
    Sir Veza and Gast86 like this.

Share This Page