Introducing AskSteem - A steem search engine
Hello, Steemians!
Over the past month, I've been building a new search engine that indexes the steem blockchain. It's currently live at asksteem.com. The goal of AskSteem is to provide a reliable, powerful, and fast search engine that is optimized for steem. In this post, I'd like to cover some of the features that are available.
Query Syntax
There are many different ways that you can query the AskSteem index. I've created a video demonstrating each of them, but you may also read their descriptions and examples below.
Keyword/Phrase Search
Like many other search engines, you can search for general phrases and terms. AskSteem will try its best to find the document that is most relevant to your query based on our ranking algorithm.
Example Queries:
Tip: Click the example to go to that query on AskSteem
How to buy bitcoin
What is steem
Markdown tutorial
Exact Search
Putting a query into quotes requests that AskSteem only returns documents that have exactly that phrase in that order.
Example Queries:
"How to buy bitcoin"
"What is steem"
"Markdown tutorial"
Tag Search
AskSteem allows you to filter posts by tag.
Example Queries:
tags:life
tags:steemit
Author Search
You can filter posts by the author too.
Example Queries:
author:thekyle
author:abit
author:steemit
Creation Date Search
AskSteem provides a highly flexible and powerful date search tool for posts. You can search by exact date or by date range. Dates must be in the form of YYYY-MM-DD.
Example Queries:
Search for all posts posted on June 2, 2017
created:2017-06-02
Search for all posts posted between May 1, 2017, and May 31, 2017
created:[2017-05-01 TO 2017-05-31]
Search by Number of Votes/Comments
Similar to dates AskSteem has another set of robust tools that allow searches based on the number of upvotes or comments a post receives.
Example Queries:
Posts with 150 votes:
net_votes:150
Posts with between 100 and 150 votes
net_votes:[100 TO 150]
Posts with 50 comments:
children:50
Posts with between 40 and 50 comments:
children:[40 TO 50]
Posts with more than 50 comments:
children:>50
or less than 50:
children:<50
this also works with votes, less than or equal to 10 votes:
net_votes:<=10
Searches with Boosts
You can prioritize certain parts of your query with boosts. These are indicated by placing a ^n at the end of a term, where n is the power you want to boost that part of the query to.
Example Queries:
Give the term bitcoin a boost of two:
I really want posts to have the term bitcoin^2 in them.
Give the term mine a boost of two, and term steem a boost of three:
How to mine^2 steem^3
Inclusive/Exclusive Search
You can indicate whether you want documents to contain certain terms by placing a + or a - in front of the term.
Example Queries:
Find documents about mining but not bitcoin:
cryptocurrency +mining -bitcoin
Wildcard Search
You can use the wildcard expressions of ? for a single character, or * to match any number of characters.
Example Queries:
How to mine any cryptocurrency:
How to mine *
Boolean Search
AskSteem supports any combination of the previously mentioned search types in a single powerful query. This uses boolean values of AND, OR, and NOT, along with parenthesis to separate statements.
Example Queries:
Posts tagged with asksteem by @thekyle:
tags:asksteem AND author:thekyle
Posts with between 50 and 100 comments that have more than 500 upvotes and that are tagged with 'bitcoin' or have the term bitcoin in the document:
(bitcoin OR tags:bitcoin) AND (net_votes:>500 AND children:(>50 AND <100))
Posts created on June 2, 2017, with 100 or more upvotes but less than 10 comments:
created:2017-06-02 AND net_votes:>=100 AND children:<10
Developers
Because AskSteem integrates directly into the steem blockchain it can read metadata directly from posts and use that data when performing queries and displaying results. We encourage developers to add AskSteem compatible metadata to their posts so that we can show links to your application in our search results. The full documentation can be found at asksteem.com/developers, however, in this post I will summarize the most important tags.
| Tag | Description | Example |
|---|---|---|
| domain | The domain name or web address that your application is hosted on. | example.com |
| locator | The path to reach the post on the domain relative to the root. | /CATEGORY/@AUTHOR/PERMLINK |
| protocol | Either 'http' or 'https' if not provided then http will be used by default | https |
If none of the above metadata is provided then AskSteem will link to steemit.com for all posts by default, however, it is assumed that the platform creating the content will have the best interface for viewing it, so we would rather link there.
The domain and locator tags are required for custom linking to work, however, the protocol tag is optional and will default to http.
- The domain tag should be the domain name that your web-based steem application is hosted on and is subdomain sensitive (so if your hosting on www subdomain then put that).
- The locator should be the permalink to that particular post in your applications URL structure, also notice the leading forward slash, this is required.
The final URL that we point to will be generated by concatenating the domain and locator together with the protocol at the beginning which will be http unless otherwise specified.
Additionally, if you are building an application on the steem blockchain and need a search API please email us at [email protected], we are able to query custom metadata and make various other customizations to the ranking algorithm to support your use case.
Funding
The harsh reality is that search engines are expensive to run and that adding new features and improving performance are difficult if the basic funding needs of the project are not covered. AskSteem currently costs me about $100/month to run, and that number will, of course, continue to increase as steem grows and the index size increases.
Ideally, my goal is the have those costs covered through upvotes from the steem community and to use any extra money for adding new features and scaling the search infrastructure to meet demand. If this works then AskSteem will be the first search engine in the world to use a cryptocurrency based revenue model, instead of selling advertising.
Thank you for your time, and happy searching!
Happy to see this. I resteemed.
Thanks, I look forward to making it even better!
yay
How do your 'Resteem' a post?
Highly recommended tags for such post #steemdev and #steem-project
You almost missed out on 300$ of reward by not having this tag.
Well thanks for letting me know, I'll make sure to include those in future posts.
you can edit
Done!
This is something many were waiting for! Tried and it works well! It has to be developed, congrats for the project!
Yes thank you! I can now search then click back after reading an article and it will take me back to my search results. Small thing but needed! 😎
Glad you like it!
Hi @thekyle.
I'd like to tell you how happy I am for the lesson, not just on asksteem.com, but also on how to do a search and ferret out what might otherwise be obscure - but important - results. I've been doing a little research now and then on a couple of open source investigations, and just learning more about how searches operate will help me to research for my own topics. It will also help me better assist the activists and freedom-fighters who are busy helping us reclaim our democracy.
It's a pleasure to meet you. I'm @joanroberts, and I've been here for a whole week. Forgive me for not upvoting - my (literal) 2 cents is going to go to someone else, who is new, but who, like yourself, adds to the community. Trust me. When your vote is only worth 2 cents, when you see someone upvote you and you've got 4 cents, while it doesn't make you rich, it does make you smile.
Hi Joan thanks for your comment. I'm glad you're enjoying AskSteem and I hope you'll continue to use it for your investigations. Have a 23 cent upvote from me. :)
@thekyle, you just brought tears to my eyes. Thank you.
Great tool, thanks for building this. Is there a way to sort the search results? For examply by number of votes or comments? Or does anybody know another tool that does this?
Not currently through the Web UI, but that functionality is available in our developer API so it may come in the future.
Ah great, you have an API. Thanks for answering, I'll definitely look into that.
@thekyle please repost this for the benefit of those who haven't had a chance to see it. I wanted to resteem but that option is gone as the post is too old. Old posts can still be a valuable resource for us on Steemit.
I'm not going to repost because I don't want to put duplicates in my feed, however, feel free to resteem this newer post about user search functionality if you'd like to spread the word.
Great post! resteem! I hope one day it will have half of google traffic :)
Good work!!! Followed, up-voted and re-steemed the post for more visibility! I hope they can add it here on Steemit and replace the searcher we have now, it is really bad.
Yeah, I would definitely love to see some steem clients (including steemit.com) start using AskSteem as their search backend.
That would be perfect! :) Nice work man!
This is good. Got a question though: Is there a way to check who resteemed a post?
No, and sadly its not likely to be added in the future because resteems are not part of the blockchain and are kept in steemit.com's private database I have no way to index them.
Too bad :/ Thanks for the quick answer.