Crypto Public Library — Needs Help w Data Processing\Entry [Up to 50 Steem in Rewards]

in utopian-io •  2 months ago


New Projects

TLDR; Creating a collaborative web-directory of "source-grade" content. For writers, students, and research nerds to easily find source-grade crypto content. To do so we will first transform a discord server, full of crypto resources organized by topic, to a database that will power our web-directory.

The Library

At, there are thousands of links organized categorically on an ever expanding array of crypto-related topics. Becoming familiar with the many facets of crypto is a monumental task. So much more difficult, because quality information can be difficult to find without quite a bit of background research. The library is a result of digging through search results to find the most informative content on given topics. When writing freelance, I commonly go through as many as 50 different links comparing different sources and cataloging any information I know will be useful later. Twitter and Reddit are also valuable sources of information, so much of it comes up organically.

We’ve started by focusing on fundamentals; general crypto topics; with a preference towards projects that are oldest, most dominant, respected, influential, etc. There are many new projects, but plenty of other research groups are focused on ICOs. The plan is to eventually cover all types of crypto knowledge, along with all of the top (somenumber) crypto projects. Seeking the highest quality information on projects regardless of our personal opinions on them, along with the highest quality criticism regardless of our personal opinions.

There are currently 125 different channels, with thousands of resources, each devoted to a particular aspect of crypto. Obviously knowledge can't be so easily compartmentalized, but we do our best. We also happen to have the largest, freely-available, collection of learning materials about trading and evaluating crypto projects than can be found anywhere else. If you're interested in that type of thing.

*Note, these channels are not all filled, but there has been significant progress towards the end goal.

Library Directory:
General Knowledge

#101 #history #satoshi #wallets #exchanges #cypherpunks #icos #evaluation #economics #governance #trading #mining #regulation #wallets #growth-adoption #traditional-finance


#resources #assorted-media #twitter-follows #communities #news-sites #swag #women #earn-crypto #conferences #books #research-papers #non-english #data


#cryptography #decentralization #privacy #security #protocols #hash-algos #open-source #forks #smart-contracts #dapps #sidechains #master-nodes #databases #ipfs #tor #pi-rasberry-etc #bittorrent #oracles #interoperability

Distributed Consensus

#distributed-systems #consensus #pow #pos #tendermint #hashgraph


#discussion-dev #hackathons #gen-developing #blockchain-developing #forking #bitcoin-dev #ethereum-dev #state-channels #id-dev #corda-dev #steem-dev #hyperledger-dev #quant-trading


#gen-crypto #bitcoin #btc-layer2 #ethereum #eth-layer2 #stable-coins #doge #zcash #cardano #stellar #ripple #tron #bch #dash #litecoin #monero #iota #neo #etc #steem #eos #bitshares #nano #hyperledger #corda #etcetera #bytom #tezos #cosmos #enterprise-dlt


#decentralized-id #finance #payment-processing #institutional-derivitives #prediction-markets #dao #fiat-onramp #publishing #united-nations #government #voting #supplychain #social-media #music #art-collectibles #iot #ai #intellectual-propery #games #energy #services #marketplaces #notary #security-tokens #law-breakers #bots

The Challenge and the Big Plan

In case it hadn't occured to you, having resources organized in 100+ discord channels isn't the most user-friendly way to run a library. It worked great when there were half as many channels, but as the library grows we need more flexibility.

The big plan is to put all of these resources into an easily navigable web-application, so that individual resources can have multiple tags; each tag can act as it's own individual directory, and multiple groupings of tags can act as their own directory. In this way, users will be able to navigate the library manually, and be able to filter search results by tag. Generally the idea is to make the library more accessable with search functionality, tagging, comments, reactions and other user activity..

A dynamic directory of crypto knowledge, and a discord bot so that you can query the library without having to scroll through channels or visit the web-site :)

The Problem and our Process

The problem is that developers don't have much to build until there is a database. In order to create the database to power a dynamic web-portal, we are using the TOML format to organize the information and describe it and tag it. TOML is a simple format that we are using to prepare our information to be transformed into a mongo database. By labeling each link with a Title, Description, and Tags we will have made it easy to be used for a web directory. Of course all of that takes a lot of work. Its easy, relaxing, and a wonderful educational opportunity, but time consuming.

We used a discord server history scraper to create 'crypt0library_history.txt' which can be viewed in

The first step is to use that viewer app to copy\paste channel histories for channels that have not had files made for them yet. The best thing would be if someone could make files for all of the channels not done yet, so we could remove one more step from our process. It's fine to copy paste each channel into an individual file with the same filename as channel name.

Those files can be added to

The next step is transforming each file of resources into toml format. (


["The **Title** of the entry and must always be different from previous entries"] 
Link = [""]
Description = "Description can be whatever text is valuable to communicate.\nTypically copied directly from the source; sometimes requiring creativity. "
Tags = ["blah", "blah-blah"]
  • For any label that can potentially have multiple values it must always be ["bracketed"] even though sometimes there is only one value.
  • I made "Link" a bracketed ["array"] so that sometimes a supporting link can be included. occasionally this will be helpful.
  • Also, all entries have to have the same fields and format
  • If you use \n for a newline, you can't actually use a newline, the open " and closing " must be on the same line
  • Be careful of quotation marks in text copied descriptions. Change to ' or use an escape character \" in front of the quotation mark
  • TOML is Case Sensitive. All the tags should be lower case. All the value names must always begin with a capital and be identical

The result of this process, once it's landed on github ^^^

Our current plan for completing population of toml files is broken down as follows:

* note: if you're volunteering to help I will be happy even if you follow your own system for completing the work. This is the best system that I could to come up with.

  1. First get the history of every channel into individual files on github.
    • simply copy\paste
  2. Clean up each of those files and begin creating the toml format like above, except only with the link field filled out, leaving the rest blank, but ready to be filled out (for the whole server).
    • a script could probably assist in this matter, but that's not my specialty
  3. then filling out the description and tags and title for each, manually.
  4. While the database is being prepared, developers are working on the web-application and a discord bot but there isn't much for them to work on without a sufficiently populated database.

The work was ordered in those steps both to get the easiest most time consuming tasks out of the way first, and to give us librarians more time to come up with a proceedure and organizaed thought around how they should be tagged. If you want to help by automating any part of that process would, it would be appreciated.

Steem Rewards

While there is no specific "per-task" rewards system layed out, we do have 50 steem set aside for the most active contributors. We are open to negotiation if you have your own idea of how much you would like for a particular task.

Visit the #help-wanted channel of to discuss further.

GitHub Account

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Thanks for the task request, @inquiringtimes. The task itself seems pretty clear to me. Good luck finding someone who can solve it for you!

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.

Need help? Write a ticket on
Chat with us on Discord.


Thank you for your review, @amosbastian! Keep up the good work!

Hey, @inquiringtimes!

Thanks for contributing on Utopian.
We’re already looking forward to your next task request!

Get higher incentives and support!
Simply set as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord

Vote for Utopian Witness!

Hi @inquiringtimes!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Congratulations @inquiringtimes! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You made more than 14000 upvotes. Your next target is to reach 15000 upvotes.

Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word STOP

Support SteemitBoard's project! Vote for its witness and get one more award!