I can't believe I made it this far !!!

in developer •  26 days ago

iteration 1
nodejs/mongo db

abysmal failure

but that's ok i guess
i did my initial assessment / proof of concept using a new platform (and landed a dev gig - pure dumb luck there)

i did pick up a little nodejs/pure javascript stuff so it wasnt a total loss

so it was only half a failure i guess

iteration 2
port everything to php / mysql

got it rocking fairly fast

my old multi-curl scraper was broken - i had fixed it / recoded it once before
but some asshole stole my dev computer that had good code on it

so i lost like 3 or 4 days troubleshooting and fixing it again

that turned out pretty good - because now it actually works - and much better than my older fixes

iteration 3

trying to fix up the loose ends

making a half decent UI is kicking my butt - i am definitely not a front-end guy LOL

problems
    using older bootstrap version with an even older bootstrap css 
    roflmao - no wonder everything i tried looked horrible
    
    i had to rip through all my old crap code and get everything up to date for 
    jquery / bootstrap
    
    wow - amazing - everything actually works like it should for once
    
    next problem - 
    i forgot to update my MAX_FILE_SIZE upload in php.ini settings      
    and that still wasnt working - 
    duh - forgot to restart my apache service so the setting weren't active
    so all my uploads still weren't working
    
    yay - problem solved after apache reboot - im such an idiot

here we are in the home stretch

i have the command line interface running awesome fast

now that i have the right UI / jquery / bootstrap / uploader stuff working i should be able to
just plug everything in to the UI fairly quick from this point.

i cant believe some of the stupid problems i have been having with this

on a positive note - all that broken stuff that has been slowing me down is fixed fixed fixed ;)

upcoming @todo list

  1. multi engine
    currently the scraper just does one site at a time - in succession

    i can get this to run a little better if i spread scraper across all domains

    i did it once before - so shouldn't be too hard - just need my brainz to cooperate

  1. ajax api call
    damn NDA - can't talk about this part

  2. finish the UI - get stats and charts working again
    what good is all this data if nobody can get to it or use it

    I probably don't have to do all the 'ajaxy' stuff -
    but what the hell , it just adds to the fun of the challenge...

    part of this is adding in some more documentation (the database design)

once again - I am going to shoot for monday launch - this should be interesting

crap - lost another 3 hours fighting the damn dev server
apparently it was still running IIS
so i had to jumble turning it off
then the friggin apache didnt want to load php - so i had to battle that crap
oh look - let's fix the firewall crap while were at it LOLZ

Well, i hope i can get this thing up and running in the next week...

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!