Feedback-based Update

This is an update for steem-exif-spider-bot

I want to give a shoutout to some people. @salty-mcgriddles and @pfunk (and anyone else I interacted with that gave some decent feedback). @salty-mcgriddles decided to run this bot for some a/b testing. The bot ran less than 24 hours. Even when it did some unexpected things @salty-mcgriddles resolved to keep it running anyway for the sake getting real data and feedback, and to compile that data into something useable.

There was a lot of mixed feedback. There was definitely a spectrum. Most of it was good. Some thought it was spammy. Only a couple really hated it (it's the internet, you're bound to encounter trolls. Personally, I'm surprised there were only 2.) As I said in another post, bots are spam machines. It doesn't matter how useful the bot is, output is always some kind of spam. There was a lot of really good feedback on how to mitigate the spam. @salty-mcgriddles suggested profiles to reduce the amount of data according to content. The trouble is, I had no way to reliably do that. Enter @pfunk. @pfunk had some really good ideas about how to get around this. I decided to implement an opt-in approach similar to how Telegram manages bots. You basically pass commands to it. That too can be pretty spammy, but I figure if someone only needed one command, then that would be fine.

Opt-in Feature

There are a lot of bots out there that already do this, so it is already intuitive to steemians. It seems like a good way to go about it. Users can simply call the bot by name from a comment on the post.
This is useful for the following reasons

1. The post is already up and the image content is live making it possible for the bot to parse it
2. Once the data is posted, the comment can be deleted leaving cleaner, more meaningful content

Here's an example of the optin feature

@exifr

This will summon the bot to leave behind some default exif data.

Profiles feature

Even if you're manually opting in, there's a lot of data and I wanted users to be able to pick and choose what gets shown. For example, users may have differing intents. Some may only want copyright information
to prove origination. Some may just minimal data. Some may have a ton of photos in their post and they only want to show what's different between the photos rather than show the same data repeatedly. All
of these things can be done. Here are some examples

Minimal Profile

@exifr minimal

Produces about 14 lines of EXIF data. Just the basics for photography. Camera info, shutter speed, aperture, focal length, flash, etc...

Basic Profile

@exifr

This is the default profile. It's a bit more liberal than the minimal profile. It's about 18-20 lines of EXIF data including details about the image like it's dimensions and what editing software was used.

Copyright

@exifr copyright

Produces about 3-4 lines of EXIF data. It just exposes the copyright details for each photo if there are any.

This is useful for checking if your images has copyright information attached. If it does not, you can remove the photo and the @exifr comment, fix the image, re-upload it, and summon the bot again.

Custom profiles

@exifr Make Model DateCreatedOriginal Software

This will give just 4 lines of EXIF data to expose some specific details about a photo. If you have more than one photo, this can get spammy because it's likely the information is not different for each photo.

@exifr ShutterSpeedValue ExposureBiasValue FocalLength LightSource FNumber ApertureValue BrightnessValue

This is actually a better example because all of these values are likely to be different for each photos.

Change Details

diff --git a/steem-exif-spider-bot/README.md b/steem-exif-spider-bot/README.md
index 974c13c..ff08913 100644
--- a/steem-exif-spider-bot/README.md
+++ b/steem-exif-spider-bot/README.md
@@ -12,6 +12,8 @@ Once [Docker](https://www.docker.com/community-edition#/download) is installed,
 |-------|-----------|
 |`STEEM_NAME`|Steem user name|
 |`STEEM_WIF`|Steem private posting key|
+|`VOTING_WEIGHT`|Integer between 0-10000 describing the percentage weight of the vote applied|
+|`STEEM_BENEFICIARIES`|Comma-separated list of steem names for beneficiaries to share stake|
 
 > **Notice** If you do not know your private posting key, it can be retrieved at: https://steemit.com/@<STEEM_NAME>/permissions
 
@@ -19,5 +21,5 @@ Once [Docker](https://www.docker.com/community-edition#/download) is installed,
 When [Docker](https://www.docker.com/community-edition#/download)  is installed and the environment variables are set, installation and execution are a single docker command: 
 
-docker run --rm -e STEEM_NAME=$STEEM_NAME -e STEEM_WIF=$STEEM_WIF r351574nc3/steem-exif-spider-bot:latest
+docker run --rm -e STEEM_NAME=$STEEM_NAME -e STEEM_WIF=$STEEM_WIF -e VOTING_WEIGHT=300 -e STEEM_BENEFICIARIES=notallorder,cutemachine r351574nc3/steem-exif-spider-bot:latest

diff --git a/steem-exif-spider-bot/config/index.js b/steem-exif-spider-bot/config/index.js
index 25fb7cf..6fce4a4 100644
--- a/steem-exif-spider-bot/config/index.js
+++ b/steem-exif-spider-bot/config/index.js
@@ -2,6 +2,7 @@ module.exports = {
     user: process.env.STEEM_NAME,
     wif: process.env.STEEM_WIF,
     weight: parseInt(process.env.VOTING_WEIGHT), 
+    bennies: process.env.STEEM_BENEFICIARIES ? process.env.STEEM_BENEFICIARIES.split(",") : [],
     blacklist: [
         "emwalker",
         "tomlabe",

diff --git a/steem-exif-spider-bot/helpers/bot/comment.js b/steem-exif-spider-bot/helpers/bot/comment.js
index 4774941..44f9e96 100644
--- a/steem-exif-spider-bot/helpers/bot/comment.js
+++ b/steem-exif-spider-bot/helpers/bot/comment.js
@@ -1,6 +1,6 @@
 const Promise = require('bluebird')
 const steem = require('steem')
-const { user, wif, weight } = require('../../config')
+const { user, wif, weight, bennies } = require('../../config')
 const schedule = require('node-schedule')
 const Handlebars = require('handlebars')
 const fs = Promise.promisifyAll(require('fs'))
@@ -13,6 +13,15 @@ function loadTemplate(template) {
     return fs.readFileAsync(template, 'utf8')
 }
 
+function get_beneficiaries() {
+    const num_beneficiaries = bennies.length
+    return bennies.map((beneficiary) => {
+        return {
+            account: beneficiary,
+            weight: 5000 / num_beneficiaries
+        }
+    });
+}

For beneficiary support. Parses configuration into a beneficiary object

 function execute(comments) {
     schedule.scheduleJob(MINUTE, function() {
@@ -27,41 +36,52 @@ function execute(comments) {
         }
 
         Promise.each(tags, (tag, index, length) => {
-            if (tag.name == 'URL') {
+            if (tag.name.toLowerCase() == 'url') {
                 context.url = tag.description
                 tags[index] = {}
             }
         })

Removing url information from the tags, so it doesn't post the image twice.

         .then(() => {
-        return loadTemplate(path.join(__dirname, '..', 'templates', "exif.hb"))
-            .then((template) => {
-                var templateSpec = Handlebars.compile(template)
-                return templateSpec(context)
-            })
-            .then((message) => {
-                var new_permlink = 're-' + author 
-                    + '-' + permlink 
-                    + '-' + new Date().toISOString().replace(/[^a-zA-Z0-9]+/g, '').toLowerCase();
-                console.log("Commenting on ", author, permlink)
-
-                return steem.broadcast.commentAsync(
-                    wif,
-                    author, // Leave parent author empty
-                    permlink, // Main tag
-                    user, // Author
-                    new_permlink, // Permlink
-                    new_permlink,
-                    message, // Body
-                    { tags: [], app: "steemit-exif-spider-bot/0.1.0" }
-                ).then((results) => {
-                    console.log(results)
-                    return results
+            var new_permlink = 're-' + author 
+                + '-' + permlink 
+                + '-' + new Date().toISOString().replace(/[^a-zA-Z0-9]+/g, '').toLowerCase();
+            console.log("Commenting on ", author, permlink)
+            return loadTemplate(path.join(__dirname, '..', 'templates', "exif.hb"))
+                .then((template) => {
+                    var templateSpec = Handlebars.compile(template)
+                    return templateSpec(context)
                 })
-                .catch((err) => {
-                    console.log("Error ", err.message)
+                .then((message) => {
+
+                    return steem.broadcast.commentAsync(
+                        wif,
+                        author, // Leave parent author empty
+                        permlink, // Main tag
+                        user, // Author
+                        new_permlink, // Permlink
+                        new_permlink,
+                        message, // Body
+                        { tags: [], app: "steemit-exif-spider-bot/0.1.0" }
+                    ).then((results) => {
+                        console.log("Comment posted: ", results)
+                        const extensions = [
+                            [
+                                0,
+                                {
+                                    beneficiaries: get_beneficiaries()
+                                }
+                            ]
+                        ];
+                        return steem.broadcast.commentOptionsAsync(wif, user, new_permlink, "1000000.000 SBD", 10000, true, false, extensions)
+                            .then((results) => {
+                                console.log("Comment Options: ", results);
+                            });
+                    })
+                    .catch((err) => {
+                        console.log("Error ", err.message)
+                    })
                 })
             })
-        })
     })
 }

This is a really long way to say, that we added comment options after a comment is posted to include beneficiary assignment.

diff --git a/steem-exif-spider-bot/helpers/bot/exif.js b/steem-exif-spider-bot/helpers/bot/exif.js
index e71d96b..fa2ed46 100644
--- a/steem-exif-spider-bot/helpers/bot/exif.js
+++ b/steem-exif-spider-bot/helpers/bot/exif.js
@@ -20,10 +20,137 @@ module.exports = {
 let VOTING = {}
 let COMMENTS = {}
 
+const exif_profiles = {
+    basics: (key) => {
+        return [
+                "make",
+                "model",
+                "pixelxdimension",
+                "pixelydimension",
+                "focallength",
+                "lightsource",
+                "flash",
+                "fnumber",
+                "exposuretime",
+                "datetime",
+                "isospeedratings",
+                "exposurebiasvalue",
+                "exposuremode",
+                "whitebalance",
+                "meteringmode",
+                "software",
+                "exposureprogram",
+                "datetimeoriginal",
+                "shutterspeedvalue",
+                "aperturevalue",
+                "brightnessvalue",
+                "focallengthin35mmfilm",
+                "creatortool"       
+            ].includes(key.toLowerCase())
+    }, 
+    minimal: (key) => {
+        return [
+                "make",
+                "model",
+                "focallength",
+                "lightsource",
+                "flash",
+                "fnumber",
+                "exposuretime",
+                "datetime",
+                "isospeedratings",
+                "exposurebiasvalue",
+                "whitebalance",
+                "meteringmode",
+                "datetimeoriginal",
+                "shutterspeedvalue" 
+            ].includes(key.toLowerCase())
+    }, 
+    copyright: (key) => {
+        return (key.indexOf("opyright") > -1
+                || key.indexOf("reator") > -1)
+    },
+    the_works: (key) => { return true}
+}
+
 function loadTemplate(template) {
     return fs.readFileAsync(template, 'utf8')
 }
 
+function get_profiles_from(text) {
+    return text.replace("@" + user, "").replace(",", " ").split(" ")
+        .filter((profile) => profile && profile.trim() != "")
+        .map((profile) => { return profile.toLowerCase()});
+}
+
+function profile_check(profiles, key) {
+    // Default profile
+    if (profiles.length < 1) {
+        return exif_profiles.basics(key)
+    }
+
+    if (profiles.includes(key.toLowerCase())) {
+        return true
+    }
+
+    // all profiles exist
+    const profile = profiles.shift();
+    if (Object.keys(exif_profiles).includes(profiles)) {
+        return exif_profiles[profile](key)
+    }
+
+    return false
+}

This is all the profile stuff. We added the different profiles, a function for fetching profiles from the body of a comment and checks to see if a profile should be applied or not.

+function handle_exif(comment, profiles, images) {
+    console.log("Handling exif for ", images)
+    return Promise.map(images, (image, index, length) => {
+        const buffers = [];
+        return got(image, {encoding: null })
+            .then((response) => {
+                const tags = ExifReader.load(response.body);
+                tags.URL = { description: image, value: image }
+                return tags;
+            })
+            .catch((error) => {
+                if (error.message == "No Exif data") {
+                    console.log("error ", error)
+                }
+                else {
+                    console.log("error ", error)
+                }
+            });
+    })
+    .filter((tags) => tags ? true : false)
+    .each((input) => {
+        const tags = []
+        let URL = ""
+
+        if (!Object.keys(input).includes("Make")) {
+            return 
+        }
+
+        for (let key in input) {
+            const value = input[key];
+            if (key == "URL") {
+                URL = value
+                tags.push({ name: key, value: value.value, description: value.description })                
+            }
+
+            if (profile_check(profiles, key)) {            
+                tags.push({ name: key, value: value.value, description: value.description })
+            }
+        }
+        console.log("Pushing tags for ", URL)
+        reply(comment, tags)
+    })
+    .catch((error) => {
+        console.log("Error ", error)
+    });
+
+}

Added check for profiles. Refactored exif parsing code into its own function.

 function processComment(comment) {
     return steem.api.getContentAsync(comment.author, comment.permlink)
         .then((content) => {
@@ -33,71 +160,52 @@ function processComment(comment) {
             return {};
         })
         .then((metadata) => {
-            if ((metadata.tags && (metadata.tags.includes("photofeed")
-                                    || metadata.tags.includes("photography")))
-                && metadata.image && metadata.image.length > 0) {
-                return metadata.image;
-            }
-            return [];
-        })
-        .map((image) => {
-            if (image.indexOf(".jpg") > -1 || image.indexOf(".JPG") > -1) {
-                const buffers = [];
-                return got(image, {encoding: null })
-                    .then((response) => {
-                        const tags = ExifReader.load(response.body);
-                        tags.URL = { description: image, value: image }
-                        return tags;
-                    })
-                    .catch((error) => {
-                        if (error.message == "No Exif data") {
+            if (metadata.users && metadata.users.includes(user)) {
+                console.log("Found @exifr request")
 
+                if (comment.parent_author == "") {
+                    metadata.exif_profiles = get_profiles_from(comment.body)
+                    return metadata;
+                }
+                
+                return steem.api.getContentAsync(comment.parent_author, comment.parent_permlink)
+                    .then((content) => {
+                        if (content.json_metadata && content.json_metadata != '') {
+                            const metadata = JSON.parse(content.json_metadata);
+                            metadata.exif_profiles = get_profiles_from(comment.body)
+                            return metadata;
                         }
-                        else {
-                            console.log("error ", error)
-                        }
-                    });
+                        return {};
+                    })
             }
+            return {}
         })
-        .filter((tags) => tags ? true : false)
-        .each((input) => {
-            const tags = []
-            let URL = ""
-            for (let key in input) {
-
-                const value = input[key];
-                if (key == "URL") {
-                    URL = input[key]
-                }
-
-                if (key != "MakerNote"
-                    && key.indexOf("undefined") < 0
-                    && key.indexOf("omment") < 0
-                    && key.indexOf("ersion") < 0) {
-                    tags.push({ name: key, value: value.value, description: value.description })
-                }
+        .then((metadata) => {
+            if (metadata.image && metadata.image.length > 0) {
+                return [metadata.exif_profiles, metadata.image]
             }
-            if (tags.length > 5) {
-                console.log("Pushing tags for ", URL)
-                reply(comment, tags)
+            return [];
+        })
+        .spread((profiles, images) => {
+            if (images) {
+                return handle_exif(comment, profiles, images);
             }
+            return {}
         })
-        .catch((error) => {
-            console.log("Error ", error)
-        });
 }

As part of a refactoring, a whole bunch of code was removed. Some code was refactored into a separate function handle_exif. Implemented profile parsing here to see if they needed to be applied. There was a check added for manual opt-in trigger.

 function reply(comment, tags) {
-    const context = {
-        poster: comment.author,
-        tags: tags
+    const target = { author: comment.author, permlink: comment.permlink }
+    if (comment.parent_author != "") {
+        target.author = comment.parent_author
+        target.permlink = comment.parent_permlink
     }
 
     return new Promise((resolve, reject) => {
         // console.log("Pushing comment for ", { author: comment.author, permlink: comment.permlink})
-        COMMENTS.push({ author: comment.author, permlink: comment.permlink, tags: tags })
+        COMMENTS.push({ author: target.author, permlink: target.permlink, tags: tags })
 
-        resolve([ comment.author, comment.permlink])
+        resolve([ target.author, target.permlink])
     })
     .spread((author, permlink) => {
         console.log("Pushing vote for ", { author: author, permlink: permlink, weight: weight })
@@ -123,9 +231,7 @@ function execute(voting, comments) {
         .spread((operation_name, operation) => {
             switch(operation_name) {
                 case "comment":
-                    if (operation.parent_author == '') {
-                        return processComment(operation);
-                    }
+                    return processComment(operation);
                 break;
                 default:
             }

This was refactored to take into consideration if the opt-in implementation is supported within the body of a post (not just comments)

Roadmap

• Specific photo targeting (if you only want details for one or more photos)
• Natural language processing commands. This will allow querying the bot to analyze photos and EXIF data to determine
  • better tags for the post
  • smarter EXIF data reporting
  • Human readable (instead of a table) output

---

Posted on Utopian.io - Rewarding Open Source Contributors

---