Live SteemMonsters JSON data to Google Sheets

in #howto6 years ago

Further to my previous post about working with the @SteemMonsters data in a Google Sheets spreadsheet, and in light of my previous howto guide for connecting LibreOffice to the live data from @Steempunknet, I thought I might have a go at combining these two techniques into one, and bring the live JSON data from #SteemMonsters into a Google Sheets spreadsheet this time. The first thing I had to do was establish that there were publicly available JSON endpoints for SteemMonsters, as there are for Steempunk.net, so I checked this with @Yabapmatt, and he confirmed that there are, and gave me the details for the endpoint I'll need for the cards. There are other JSON endpoints exposed by SteemMonsters, but this is the only one we'll be dealing with in this #howto guide. The endpoint I'm using is steemmonsters.com/cards/get_details.

❶ Locating a script that can import JSON

The next thing that I had to do is discover how to access JSON data using Google Sheets. As you will see from my previous howto guide, which dealt with LibreOffice Calc, it isn't always a straightforward process and can involve some script installation first. For Google Sheets, I came across this post on Medium by Paul Gambill, and the related GitHub Gist for import_json_appsscript.js. There is also further documentation for this script here.

❷ Install the script

As always, you should be very careful when installing scripts, especially if you don't know the author, or don't know precisely what the script does, or if you can't read the code yourself to ascertain its validity and safety. That said, I haven't had too many problems installing scripts in Google cloud services in the past.

As you'll see from that Medium post, the instrctions to install the script in your Google Sheets spreadsheet is quite straighforward...

  1. Create a new Google Spreadsheet.
  2. Click on Tools -> Script Editor.
  3. Click Create script for Spreadsheet.
  4. Delete the placeholder content and paste the code from this script.
  5. Rename the script to ImportJSON.gs and click the save button.
  6. Back in the spreadsheet, in a cell, you can type =ImportJSON() and begin filling out it’s parameters.

That's all there is to it! Very simple to install. Next we'll look at using it to access the live data from SteemMonsters.

❸ Connect to the API

The next step is to connect to the #SteemMonsters API endpoint that you are interested in. For this example, we are going to use the /cards/get_details endpoint, as mentioned above. To do a simple test of this, we can import the entire table into our spreadsheet in one hit, with a header row, by typing the following into cell A1...

=𝙸𝚖𝚙𝚘𝚛𝚝𝙹𝚂𝙾𝙽("𝚑𝚝𝚝𝚙𝚜://𝚜𝚝𝚎𝚎𝚖𝚖𝚘𝚗𝚜𝚝𝚎𝚛𝚜.𝚌𝚘𝚖/𝚌𝚊𝚛𝚍𝚜/𝚐𝚎𝚝_𝚍𝚎𝚝𝚊𝚒𝚕𝚜","","𝚗𝚘𝚒𝚗𝚑𝚎𝚛𝚒𝚝, 𝚗𝚘𝚃𝚛𝚞𝚗𝚌𝚊𝚝𝚎, 𝚛𝚊𝚠𝙷𝚎𝚊𝚍𝚎𝚛𝚜")

When you press ENTER ⏎ the URL should resolve, and you should see an unformatted table full of the JSON data, as in the screenshot. One thing that you will notice is that the Stats get "rolled up" into a single cell per Stat, displaying as a comma-separated array within the cell. Similarly, the Abilities get "rolled up" into a single cell as well. Another thing that I have found is that the is_Starter field isn't handled very well, and the values all come through as "TRUE", which is, in fact, quite false. I think this has something to do with the script not handling boolean logic very well, but I'm not sure.

❹ Request specific data from the API

I found that the way the data is brought through in bulk like this isn't very user friendly. For one thing, you can't insert columns between the existing columns, which is something that I do all the time. So instead, I bring in one column at a time from API data, by adjusting the value of the second parameter of the API call. In the above example, our second parameter was left empty, (,"",). This time let's give it a specific query to search for within the results table. Edit what you have in cell A1 to now read...

=𝙸𝚖𝚙𝚘𝚛𝚝𝙹𝚂𝙾𝙽("𝚑𝚝𝚝𝚙𝚜://𝚜𝚝𝚎𝚎𝚖𝚖𝚘𝚗𝚜𝚝𝚎𝚛𝚜.𝚌𝚘𝚖/𝚌𝚊𝚛𝚍𝚜/𝚐𝚎𝚝_𝚍𝚎𝚝𝚊𝚒𝚕𝚜","/𝚒𝚍","𝚗𝚘𝚒𝚗𝚑𝚎𝚛𝚒𝚝, 𝚗𝚘𝚃𝚛𝚞𝚗𝚌𝚊𝚝𝚎, 𝚛𝚊𝚠𝙷𝚎𝚊𝚍𝚎𝚛𝚜")

Notice here that I've changed the second parameter from ,"", to ,"/id",. This time when you press ENTER ⏎ the URL should resolve, and you should see an unformatted single column of JSON data for the id numbers of the cards, as in the screenshot. We can now do the same for all of the columns. At this point it may be handy to have the /cards/get_details API endpoint visible in a browser window while you work, so that you can just copy & paste the field names into the second parameter of the API call, for each column in your spreadsheet. Also, rememer that the sub-fields for Stats and Abilities need to be specified in the format ,"/stats/mana", and ,"/stats/abilities", etc.

Finally, once you have all of your columns direct from the API, you can do some tricky things with them, like adding autofilters, formatting, and even converting those raw numbers into words. For this latter, I use a formula like this...

=𝙸𝙵(𝙵𝟸="𝟷","𝙲𝚘𝚖𝚖𝚘𝚗",𝙸𝙵(𝙵𝟸="𝟸","𝚁𝚊𝚛𝚎",𝙸𝙵(𝙵𝟸="𝟹","𝙴𝚙𝚒𝚌",𝙸𝙵(𝙵𝟸="𝟺","𝙻𝚎𝚐𝚎𝚗𝚍𝚊𝚛𝚢","𝙾𝚝𝚑𝚎𝚛"))))

Once they are converted to words, you can do some nice conditional formatting to make the spreasheet look pretty. Here's my example...

I play SteemMonsters, a collectible card battle game on the Steem blockchain. You can see my card collection in PeakMonsters.


cc-by-sa license The textual content of this post is licensed as a Free Cultural Work using a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license, which grants all the four freedoms listed in the definition of free cultural works, including: ① The freedom to use and perform the work; ② The freedom to study the work and apply the information; ③ The freedom to redistribute copies; and ④ The freedom to distribute derivative works. This license places three key restrictions on those freedoms: ⑤ 𝐍𝐨 𝐃𝐑𝐌 𝐨𝐫 𝐓𝐏𝐌: You must not restrict access to the work using technical measures, or otherwise attempt to impose limitations on the freedoms above; ⑥ 𝐀𝐭𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧: You must give proper attribution to the author (𝓣𝓻𝓲𝓼𝓺𝓾𝓮𝓵𝓦𝓱𝓪𝓻𝓮) and retain the license notice; and ⑦ 𝐂𝐨𝐩𝐲𝐥𝐞𝐟𝐭: You must release derivative works under an identical or similar license. The most recent copy of the text of this license notice (with markdown) can be found on Hackmd.
IMAGE CREDITS:
• The image used for the post thumbnail is a remix by 𝓣𝓻𝓲𝓼𝓺𝓾𝓮𝓵𝓦𝓱𝓪𝓻𝓮. It includes screenshots from Google Sheets.
• The image used for the horizontal rule separators throughout this post is from the #SteemMonsters divider set, created by @calumam. It is used with permission, and has been released under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.
• The image used for the SteemMonsters banner is a remix by 𝓣𝓻𝓲𝓼𝓺𝓾𝓮𝓵𝓦𝓱𝓪𝓻𝓮. It includes the Steem Monsters Logo Refresh image created by @mrgodby, which is used with permission, and has been released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. The remix also includes the "Rexxie" and "Goblin Socerer" assets from the official SteamMonsters collection. They are used with permission. The remixed image is released under the same license (CC BY-NC-SA 4.0) as the component part from @mrgodby, as per the ShareAlike terms.
• The image used for the PeakMonsters banner is a remix by 𝓣𝓻𝓲𝓼𝓺𝓾𝓮𝓵𝓦𝓱𝓪𝓻𝓮. It includes image elements from the PeakMonsters.com website. The remixed image is released under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Sort:  

Thanks for that @palasatenea. I enjoy doing this sort of thing, and I hope that it will come in useful for others. I'm thinking particularly of people like @littlescribe and @coruscate, who will be keeping track of the results of the battles and tournaments, once they begin. I hope that this type of thing might come in particularly handy for them. 👍

For them and for players like me who can't max out everything and have to be selective about which cards upgrade and how much :).

Oh, I'm in the same boat there. I've been buying up Summoners  as much as I can, because they're harder to get. I haven't been too concerned about the Commons  yet, and I'm still giving them away in my #payitforward effort to attract n00bies to the game. Once the battles and tournaments begin though, I'll have to start leveling up some of my Commons  to be able to fight with. Everything else, the Rares, Epics, and Legendaries  has been from blind luck in the few packs that I have been able to purchase, or mostly from prizes from competitions, or gifts/donations from others here in the community.

I'm not too concerned about the Gold Foils  as far as battling is concerned. I know I'll never be able to afford that many of them, so when I do get one in a booster pack or as a prize from a competition, it gets displayed for a while at a ludicrous high price in the Market, and eventually I will drop the price to sell it, to raise funds for more Regular  finish cards.

Great work there! Someone has been very busy!

Yes, someone has been very busy ... but not me, haha. This is mostly @yabapmatt and @aggroed's work. I certainly didn't type in all that data! I'm just pulling the live data in from the SteemMonsters website using the API. The busy ones are the dev team who put all that information there in the first place. 😉

Damn, this is great!
Thanks for going through all the steps you've taken to get to this point. Very helpful for anyone that wishes to get their feet wet with the Steem Monsters end-point data and JSON!

Cheers,
OFI-sig.JPG

Thanks for that @onefatindian. Yes, it is a long way from complete yet. Next I'll need to be able to access the information in those arrays, such as the Stats at each level. Also, I'd really like to know if there is any significance to the order of the Abilities in their array? Note how in the example below, the Blast  ability is in the third position for the Cerberus, but it is in the second position of the array for the Serpent of the Flames. I don't know if that is significant or not.

Hey @trisquelwhare,

I'm pretty sure your "stats abilities" column is not lined up correctly with each monster. For example, Cerberus does not have the ability 'Blast'... but the monster below, Fire Demon, does.

Cerberus has the ability 'Retaliate':
SmartSelect_20180905-231424_Chrome.jpg

Following this, the first position in an array is actually indicated as 0 (zero). So in this case, 'Retaliate' is in position 3, which coincides with Cerberus gaining this ability at level 4.

Which means, when an ability appears in the array, that monster gains that ability from that level onward.

Hope this helps!

Cheers,
OFI

Yes, you're quite right there @onefatindian, I wonder how that happened? I'll have to look into that to try to sort it out. Thanks for bringing this to my attention. At this stage I wasn't focusing on the detail of what was in the data coming via the API, I was more just chuffed that I was even able to get any  data through from it, haha. I'm really learning as I go along, here. 😎

    

Hmm, I have looked into this further, and you're not going to believe what I've discovered. My beloved Summoners  are at fault! 😓  It seems that the Summoners  don't follow the same pattern to the number of fields, so they get mucked up when the script pulls the data in from the API. Because there are no Abilities listed for Malric Inferno, and not even a placeholder for them, the script gets confused and moves everything over by one, giving Malric the Abilities from the next Monster  on the list, the Serpentine Soldier, and everything is out of whack from then on, and I presume this problem is cumulative further down, as more Summoners  are encountered by the script. 😯

I wonder if @yabapmatt and @aggroed are aware of this, and if it will cause any other problems later on down the track, or if this is just a problem that I am experiencing because of my particular choice of JSON importing script? I don't know, but I hope there is a simple solution to it.

That would make sense.

Is there no way to parse the data for "type":"Monster" only?

Really this isn't an API you're interfacing with, rather the results of an API call (something they've exposed externally). I think you'll have to get a filter going and try to go from there.

Great progress, @trisquelwhare! I like where this is heading :D

Congratulations @trisquelwhare! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

Award for the number of posts published

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Do not miss the last post from @steemitboard:

SteemFest³ - SteemitBoard support the Travel Reimbursement Fund.

You can upvote this notification to help all Steemit users. Learn why here!

Well, Hello again!!! How have you been @trisquelwhare? As you may know @mister-meeseeks featured you in this weeks @pifc's Pay It Forward contest.

Great article and how to. You really went into detail. It's the best way to teach others about certain aspects of Steemit. Keep up the great work.

Hi there @tryskele, nice to catch up with you again! Thanks for your comment, I appreciate all the feedback on this. I'm still trying to come up with a solution to the flaw in it that @onefatindian had identified above. I haven't heard anything back from @yabapmatt on this one yet. It may just be an issue with the Google Sheets script that I'm using, and has nothing at all to do with SteemMonsters.

Your post was upvoted by the @archdruid gaming curation team in partnership with @curie to support spreading the rewards to great content. Join the Archdruid Gaming Community at https://discord.gg/nAUkxws. Good Game, Well Played!

Yowzers! Another Archdruid Curie?! Thanks so much for this. Sorry it took me a while to catch up on this notification, I have been going through my notifications sequentially (that's Aspergers, haha), and I am so happy to finally be caught up to this one! This is a great honour, thank you!

Thank you for that. As you can see in the comments, I've hit a bit of a snag, and I'm not competent enough to know whether it is being caused by the SteemMonsters JSON API endpoint itself, or by the custom Google Sheets script that I have installed to connect to the API. I've notified @aggroed and @yabapmatt about it though, and I'm sure that when they're not too busy they'll take a look at it.

I am not into steem monsters but you did a very good job with great patience in detailing these instructions. Well done!

I came across your article through @mister-meeseeks who featured it in his post as an entry for the Pay It Forward Curation Contest. The contest is open to everyone so you are welcome to join.

Thanks a lot for that @macoolette. As I mentioned on @mister-meeseeks' post, I already do some #payitforward stuff, as I have received so much help and support from the #SteemMonsters community and they inspire me to give to others, just as I have been given to. I've even created a special, custom logo for anything relating to SteemMonsters and pay-it-forward (as posted in my reply on Meeseeks' post). I'd love to get involved in other ways to pay-it-forward, and I intend to start participating in your contests there. I'm now following the @pifc account, too. Thanks for your support. 👍

Oh, it is very good to know that you support the initiative of #payitforward. Though I know on the other hand that this is not the only way to pay it forward. There are many countless ways to help. Thanks for considering to join.

Congratulations @trisquelwhare! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

Award for the number of upvotes

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Do not miss the last post from @steemitboard:

SteemFest³ - SteemitBoard support the Travel Reimbursement Fund.

You can upvote this notification to help all Steemit users. Learn why here!

Reserved for future reference

Coin Marketplace

STEEM 0.31
TRX 0.11
JST 0.034
BTC 66765.98
ETH 3234.00
USDT 1.00
SBD 4.23