Scraping Websites that Don't Want to Be Scraped
Back to main pageFor one of my more recent projects, I needed to scrape a website containing a vast volume of user populated chord and song data.
The website's owners understandably didn't want to be scraped because their business model is presumably based on the ad revenue coming in from visitors on their site, and if I were to extract their data and host it elsewhere it would likely lose them money (especially considering how aggressive the website is with their adverts). That wasn't my goal, however, and the safeguards they put in place to prevent scraping just gave me a lot of headaches.
Here, I'm talking through the different methods I tried before finally breaking through their defenses and making a much more efficient scraper by exploiting a silly development decision.
Problems
No way to easily obtain the url of every song on the site (upwards of a million urls)
The unfortunate but necessary solution was to start by going through the list of artists (0-9, A, B, C, D, ...) and obtain the url to every artists's portfolio of songs, which I could then scrape for data.The page is dynamically rendered with JavaScript
This is where the most time was lost in the initial designs of the scraper.The website doesn't like getting DoSsed
I was making 10+ requests a second so something like fail2ban must have kicked in, because after about two hours of successful scraping, my IP was banned :| Thankfully, I found a solution.
Attempt 1: Curl and Python
First, I tried to scrape it as simply as possible. I identified the <href> elements that contained the urls I was interested in and by using curl, I downloaded a static page and tried to use python's html.parser to select all such elements.
Unfortunately, none of the body is populated until some script runs on the webpage, so it kept failing and I didn't get anywhere with this approach.
Attempt 2: Selenium
Shamefully, instead of looking for ways to break the page and bypass that script, my next idea was to use something like selenium to dynamically load the page and extract elements from that dynamically loaded page.
This was a horrible decision - not only was it slow, but I was trying to run multiple "threads" by making them background processes, and the CPU usage with this method frequently used the majority of my 8 cores. On the plus side, I made too few requests for any kind of DoS prevention software to kick in!
Attempt 3: Curl and Python... Again!
At this point I asked a friend at the Recurse Center what observations he might have about the problem, and lo, my problem with dynamically loading the site was suddenly solved!
He pointed out that if you look at the page you obtain with curl, it turns out to contain a peculiar script with nothing but an enormous (40,000 character!) JSON object:
window.UGAPP.store.page = {"template":{"module":"tab","controller":"view","action":"index"},"data":{"mobile_device":null,"show_rate_layer":false,"tab":{"id":1429865,"song_name":"The Mute","artist_name":"Radical Face","type":"Chords","part":"","version":1,"votes":73,"rating":4.913199999999999789679350215010344982147216796875,"date":"1384387201","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":null,"verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/radical_face_21222","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_chords_1429865","date_update":"1473070972","user_id":1881350,"user_iq":115,"username":"MagicalZebra","type_name":"Chords","song_id":1419512,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_guitar_pro_1950196"},"backing_track_tip":false,"tab_view":{"wiki_tab":{"content":"In standard tuning, lob a capo on the 6th fret. Ben is a half step down, so he'd \nhave his capo on 7th. All chords relative to capo.\n\n[Intro] \n[ch]G[\/ch] [ch]C[\/ch] (a few times with nice variants), then strum the G a bit going into the verse\n\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nWell, as a child I mostly spoke inside my head\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nI had conversations with the clouds, the dogs, the dead\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd they thought my broken, that my tongue was coated lead\n [ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\nBut I just couldn't make my words make sense to them\n [ch]D[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nIf you only listen with your ears... I can't get in\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd I spent my evenings pullin' stars out of the sky\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd I'd arrange them on the lawn where I would lie\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd in the wind I'd taste the dreams of distant lives\n [ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\nAnd I would dress myself up in them through the night\n [ch]D[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nWhile my folks would sleep in separate beds... and wonder why\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch] x3\n[ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\n[ch]D[\/ch] [ch]C[\/ch] [ch]D[\/ch]\n[ch]G[\/ch]\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd through them days I was a ghost atop my chair\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nMy dad considered me a cross he had to bear\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd in my head I'd sing apologies and stare\n [ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\nAs my mom would hang the clothes across the line\n [ch]D[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd she would try to keep the empty... from her eyes\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nSo, then one afternoon I dressed myself alone\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nI packed my pillowcase with everything I owned\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nAnd in my head I said \"goodbye,\" then I was gone\n [ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\nAnd I set out on the heels of the unknown\n [ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\nSo my folks could have a new life of their own\n [ch]D[\/ch] [ch]C[\/ch] \nSo that maybe I could find someone\n [ch]D[\/ch] [ch]C[\/ch] [ch]G[\/ch]\nWho could hear the only words that I'd known \n\n\n[ch]Em[\/ch] [ch]C[\/ch] [ch]G[\/ch] x7\n[ch]D[\/ch] [ch]C[\/ch] [ch]Em[\/ch]\n[ch]D[\/ch] [ch]C[\/ch]\n[ch]D[\/ch]\n\nThe song ends on that very contemplative D.\nTabbed by magicalzebra","revision_id":3,"user_id":19747408,"username":"tetroboylp","date":1557609740},"contributors":[{"userid":"726057","username":"Fox-x-x","usergroupid":"21","iq":84347},{"userid":"19747408","username":"tetroboylp","usergroupid":"21","iq":121}],"strummings":[{"part":"Main Pattern","denuminator":16,"bpm":112,"is_triplet":0,"measures":[{"measure":1},{"measure":202},{"measure":3},{"measure":202},{"measure":1},{"measure":101},{"measure":1},{"measure":101}]},{"part":"Variation","denuminator":16,"bpm":112,"is_triplet":0,"measures":[{"measure":1},{"measure":202},{"measure":3},{"measure":101},{"measure":1},{"measure":202},{"measure":3},{"measure":101}]}],"has_hq_description":0,"blocked":false,"adsupp_binary_blocked":null,"meta":{"capo":6,"tuning":{"name":"Half-Step Down","value":"Eb Ab Db Gb Bb Eb","index":2},"difficulty":"novice"},"has_official_version":false,"has_chord_pro":false,"has_preset":false,"versions":[{"id":1430977,"song_name":"The Mute","artist_name":"Radical Face","type":"Chords","part":"","version":2,"votes":10,"rating":4.49261999999999961374896884080953896045684814453125,"date":"1384387201","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/radical_face_21222","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_chords_1430977","date_update":"1479291040","user_id":3375644,"user_iq":63,"username":"jordanjboyd","type_name":"Chords","song_id":1419512,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_guitar_pro_1950196"},{"id":1896971,"song_name":"The Mute","artist_name":"Radical Face","type":"Chords","part":"","version":3,"votes":3,"rating":4.56735000000000024300561562995426356792449951171875,"date":"1477488224","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"Hey guys. Here is an alternative version of \"The mute\" by radical face. Versions 1 and 2 of the tab\/chords require a capo on the 6th or 7th fret. The version below does not require, necessarily, the capo, as you can achieve the same effect tuning your guitar half a tone up. That means, you can choose either to fit the capo on the first fret of your guitar or just tune it half a way up. Hope you like it.","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/radical_face_21222","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_chords_1896971","date_update":"1477488315","user_id":2332373,"user_iq":22,"username":"riquinholyra","type_name":"Chords","song_id":1419512,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/radical_face\/the_mute_guitar_pro_1950196"}],"song_image":false,"applicature":{"G":[{"id":"320003","listCapos":[],"noteIndex":7,"notes":[55,47,43,38,35,31],"frets":[3,0,0,0,2,3],"fingers":[3,0,0,0,1,2],"fret":0},{"id":"320033","listCapos":[],"noteIndex":7,"notes":[55,50,43,38,35,31],"frets":[3,3,0,0,2,3],"fingers":[4,3,0,0,1,2],"fret":0},{"id":"320403","listCapos":[],"noteIndex":7,"notes":[55,47,47,38,35,31],"frets":[3,0,4,0,2,3],"fingers":[3,0,4,0,1,2],"fret":0},{"id":"355433","listCapos":[{"fret":3,"startString":0,"lastString":5,"finger":1}],"noteIndex":7,"notes":[55,50,47,43,38,31],"frets":[3,3,4,5,5,3],"fingers":[0,0,2,4,3,0],"fret":3},{"id":"xx5433","listCapos":[{"fret":3,"startString":0,"lastString":1,"finger":1}],"noteIndex":7,"notes":[55,50,47,43,-1,-1],"frets":[3,3,4,5,-1,-1],"fingers":[0,0,2,3,0,0],"fret":3},{"id":"xx5787","listCapos":[],"noteIndex":7,"notes":[59,55,50,43,-1,-1],"frets":[7,8,7,5,-1,-1],"fingers":[3,4,2,1,0,0],"fret":5},{"id":"x109787","listCapos":[{"fret":7,"startString":0,"lastString":2,"finger":1}],"noteIndex":7,"notes":[59,55,50,47,43,-1],"frets":[7,8,7,9,10,-1],"fingers":[0,2,0,3,4,0],"fret":7},{"id":"x1012121210","listCapos":[{"fret":10,"startString":0,"lastString":4,"finger":1}],"noteIndex":7,"notes":[62,59,55,50,43,-1],"frets":[10,12,12,12,10,-1],"fingers":[0,4,3,2,0,0],"fret":10},{"id":"32000x","listCapos":[],"noteIndex":7,"notes":[-1,47,43,38,35,31],"frets":[-1,0,0,0,2,3],"fingers":[0,0,0,0,1,2],"fret":0},{"id":"32003x","listCapos":[],"noteIndex":7,"notes":[-1,50,43,38,35,31],"frets":[-1,3,0,0,2,3],"fingers":[0,3,0,0,1,2],"fret":0},{"id":"32040x","listCapos":[],"noteIndex":7,"notes":[-1,47,47,38,35,31],"frets":[-1,0,4,0,2,3],"fingers":[0,0,3,0,1,2],"fret":0},{"id":"32043x","listCapos":[],"noteIndex":7,"notes":[-1,50,47,38,35,31],"frets":[-1,3,4,0,2,3],"fingers":[0,3,4,0,1,2],"fret":0},{"id":"35543x","listCapos":[{"fret":3,"startString":1,"lastString":5,"finger":1}],"noteIndex":7,"notes":[-1,50,47,43,38,31],"frets":[-1,3,4,5,5,3],"fingers":[0,0,2,4,3,0],"fret":3},{"id":"x10978x","listCapos":[],"noteIndex":7,"notes":[-1,55,50,47,43,-1],"frets":[-1,8,7,9,10,-1],"fingers":[0,2,1,3,4,0],"fret":7},{"id":"x10121212x","listCapos":[{"fret":12,"startString":1,"lastString":3,"finger":3}],"noteIndex":7,"notes":[-1,59,55,50,43,-1],"frets":[-1,12,12,12,10,-1],"fingers":[0,0,0,0,1,0],"fret":10},{"id":"3200xx","listCapos":[],"noteIndex":7,"notes":[-1,-1,43,38,35,31],"frets":[-1,-1,0,0,2,3],"fingers":[0,0,0,0,1,2],"fret":0},{"id":"3204xx","listCapos":[],"noteIndex":7,"notes":[-1,-1,47,38,35,31],"frets":[-1,-1,4,0,2,3],"fingers":[0,0,3,0,1,2],"fret":0},{"id":"3554xx","listCapos":[],"noteIndex":7,"notes":[-1,-1,47,43,38,31],"frets":[-1,-1,4,5,5,3],"fingers":[0,0,2,4,3,1],"fret":3},{"id":"3x0003","listCapos":[],"noteIndex":7,"notes":[55,47,43,38,-1,31],"frets":[3,0,0,0,-1,3],"fingers":[2,0,0,0,0,1],"fret":3},{"id":"3x0403","listCapos":[],"noteIndex":7,"notes":[55,47,47,38,-1,31],"frets":[3,0,4,0,-1,3],"fingers":[2,0,3,0,0,1],"fret":3},{"id":"x10x787","listCapos":[{"fret":7,"startString":0,"lastString":2,"finger":1}],"noteIndex":7,"notes":[59,55,50,-1,43,-1],"frets":[7,8,7,-1,10,-1],"fingers":[0,2,0,0,4,0],"fret":7},{"id":"3x000x","listCapos":[],"noteIndex":7,"notes":[-1,47,43,38,-1,31],"frets":[-1,0,0,0,-1,3],"fingers":[0,0,0,0,0,1],"fret":3},{"id":"3x040x","listCapos":[],"noteIndex":7,"notes":[-1,47,47,38,-1,31],"frets":[-1,0,4,0,-1,3],"fingers":[0,0,2,0,0,1],"fret":3},{"id":"3x043x","listCapos":[],"noteIndex":7,"notes":[-1,50,47,38,-1,31],"frets":[-1,3,4,0,-1,3],"fingers":[0,2,3,0,0,1],"fret":3},{"id":"3x543x","listCapos":[],"noteIndex":7,"notes":[-1,50,47,43,-1,31],"frets":[-1,3,4,5,-1,3],"fingers":[0,2,3,4,0,1],"fret":3}],"C":[{"id":"x32010","listCapos":[],"noteIndex":0,"notes":[52,48,43,40,36,-1],"frets":[0,1,0,2,3,-1],"fingers":[0,1,0,2,3,0],"fret":0},{"id":"x32013","listCapos":[],"noteIndex":0,"notes":[55,48,43,40,36,-1],"frets":[3,1,0,2,3,-1],"fingers":[4,1,0,2,3,0],"fret":0},{"id":"x35553","listCapos":[{"fret":3,"startString":0,"lastString":4,"finger":1}],"noteIndex":0,"notes":[55,52,48,43,36,-1],"frets":[3,5,5,5,3,-1],"fingers":[0,4,3,2,0,0],"fret":3},{"id":"81010988","listCapos":[{"fret":8,"startString":0,"lastString":5,"finger":1}],"noteIndex":0,"notes":[60,55,52,48,43,36],"frets":[8,8,9,10,10,8],"fingers":[0,0,2,4,3,0],"fret":8},{"id":"xx10988","listCapos":[{"fret":8,"startString":0,"lastString":1,"finger":1}],"noteIndex":0,"notes":[60,55,52,48,-1,-1],"frets":[8,8,9,10,-1,-1],"fingers":[0,0,2,3,0,0],"fret":8},{"id":"xx10121312","listCapos":[],"noteIndex":0,"notes":[64,60,55,48,-1,-1],"frets":[12,13,12,10,-1,-1],"fingers":[3,4,2,1,0,0],"fret":10},{"id":"x3201x","listCapos":[],"noteIndex":0,"notes":[-1,48,43,40,36,-1],"frets":[-1,1,0,2,3,-1],"fingers":[0,1,0,2,3,0],"fret":0},{"id":"x3555x","listCapos":[{"fret":5,"startString":1,"lastString":3,"finger":3}],"noteIndex":0,"notes":[-1,52,48,43,36,-1],"frets":[-1,5,5,5,3,-1],"fingers":[0,0,0,0,1,0],"fret":3},{"id":"87555x","listCapos":[{"fret":5,"startString":1,"lastString":3,"finger":1}],"noteIndex":0,"notes":[-1,52,48,43,40,36],"frets":[-1,5,5,5,7,8],"fingers":[0,0,0,0,3,4],"fret":5},{"id":"8101098x","listCapos":[{"fret":8,"startString":1,"lastString":5,"finger":1}],"noteIndex":0,"notes":[-1,55,52,48,43,36],"frets":[-1,8,9,10,10,8],"fingers":[0,0,2,4,3,0],"fret":8},{"id":"8755xx","listCapos":[{"fret":5,"startString":2,"lastString":3,"finger":1}],"noteIndex":0,"notes":[-1,-1,48,43,40,36],"frets":[-1,-1,5,5,7,8],"fingers":[0,0,0,0,3,4],"fret":5},{"id":"810109xx","listCapos":[],"noteIndex":0,"notes":[-1,-1,52,48,43,36],"frets":[-1,-1,9,10,10,8],"fingers":[0,0,2,4,3,1],"fret":8},{"id":"x3x010","listCapos":[],"noteIndex":0,"notes":[52,48,43,-1,36,-1],"frets":[0,1,0,-1,3,-1],"fingers":[0,1,0,0,3,0],"fret":0},{"id":"8x555x","listCapos":[{"fret":5,"startString":1,"lastString":3,"finger":1}],"noteIndex":0,"notes":[-1,52,48,43,-1,36],"frets":[-1,5,5,5,-1,8],"fingers":[0,0,0,0,0,4],"fret":5},{"id":"8x1098x","listCapos":[],"noteIndex":0,"notes":[-1,55,52,48,-1,36],"frets":[-1,8,9,10,-1,8],"fingers":[0,2,3,4,0,1],"fret":8}],"Em":[{"id":"022000","listCapos":[],"noteIndex":4,"notes":[52,47,43,40,35,28],"frets":[0,0,0,2,2,0],"fingers":[0,0,0,2,1,0],"fret":0},{"id":"022003","listCapos":[],"noteIndex":4,"notes":[55,47,43,40,35,28],"frets":[3,0,0,2,2,0],"fingers":[3,0,0,2,1,0],"fret":0},{"id":"xx2000","listCapos":[],"noteIndex":4,"notes":[52,47,43,40,-1,-1],"frets":[0,0,0,2,-1,-1],"fingers":[0,0,0,1,0,0],"fret":0},{"id":"022403","listCapos":[],"noteIndex":4,"notes":[55,47,47,40,35,28],"frets":[3,0,4,2,2,0],"fingers":[3,0,4,2,1,0],"fret":0},{"id":"xx2003","listCapos":[],"noteIndex":4,"notes":[55,47,43,40,-1,-1],"frets":[3,0,0,2,-1,-1],"fingers":[2,0,0,1,0,0],"fret":0},{"id":"022453","listCapos":[{"fret":2,"startString":3,"lastString":4,"finger":1}],"noteIndex":4,"notes":[55,52,47,40,35,28],"frets":[3,5,4,2,2,0],"fingers":[2,4,3,0,0,0],"fret":0},{"id":"xx2403","listCapos":[],"noteIndex":4,"notes":[55,47,47,40,-1,-1],"frets":[3,0,4,2,-1,-1],"fingers":[2,0,3,1,0,0],"fret":0},{"id":"xx2453","listCapos":[],"noteIndex":4,"notes":[55,52,47,40,-1,-1],"frets":[3,5,4,2,-1,-1],"fingers":[2,4,3,1,0,0],"fret":2},{"id":"079987","listCapos":[{"fret":7,"startString":0,"lastString":4,"finger":1}],"noteIndex":4,"notes":[59,55,52,47,40,28],"frets":[7,8,9,9,7,0],"fingers":[0,2,4,3,0,0],"fret":7},{"id":"x79987","listCapos":[{"fret":7,"startString":0,"lastString":4,"finger":1}],"noteIndex":4,"notes":[59,55,52,47,40,-1],"frets":[7,8,9,9,7,-1],"fingers":[0,2,4,3,0,0],"fret":7},{"id":"010991212","listCapos":[{"fret":9,"startString":2,"lastString":3,"finger":1},{"fret":12,"startString":0,"lastString":1,"finger":4}],"noteIndex":4,"notes":[64,59,52,47,43,28],"frets":[12,12,9,9,10,0],"fingers":[0,0,0,0,2,0],"fret":9},{"id":"0109121212","listCapos":[{"fret":12,"startString":0,"lastString":2,"finger":4}],"noteIndex":4,"notes":[64,59,55,47,43,28],"frets":[12,12,12,9,10,0],"fingers":[0,0,0,1,2,0],"fret":9},{"id":"02200x","listCapos":[],"noteIndex":4,"notes":[-1,47,43,40,35,28],"frets":[-1,0,0,2,2,0],"fingers":[0,0,0,2,1,0],"fret":0},{"id":"07998x","listCapos":[],"noteIndex":4,"notes":[-1,55,52,47,40,28],"frets":[-1,8,9,9,7,0],"fingers":[0,2,4,3,1,0],"fret":7},{"id":"x7998x","listCapos":[],"noteIndex":4,"notes":[-1,55,52,47,40,-1],"frets":[-1,8,9,9,7,-1],"fingers":[0,2,4,3,1,0],"fret":7},{"id":"010998x","listCapos":[],"noteIndex":4,"notes":[-1,55,52,47,43,28],"frets":[-1,8,9,9,10,0],"fingers":[0,1,3,2,4,0],"fret":8},{"id":"0109912x","listCapos":[{"fret":9,"startString":2,"lastString":3,"finger":1}],"noteIndex":4,"notes":[-1,59,52,47,43,28],"frets":[-1,12,9,9,10,0],"fingers":[0,4,0,0,2,0],"fret":9},{"id":"01091212x","listCapos":[{"fret":12,"startString":1,"lastString":2,"finger":4}],"noteIndex":4,"notes":[-1,59,55,47,43,28],"frets":[-1,12,12,9,10,0],"fingers":[0,0,0,1,2,0],"fret":9},{"id":"0220xx","listCapos":[],"noteIndex":4,"notes":[-1,-1,43,40,35,28],"frets":[-1,-1,0,2,2,0],"fingers":[0,0,0,2,1,0],"fret":0},{"id":"0254xx","listCapos":[],"noteIndex":4,"notes":[-1,-1,47,43,35,28],"frets":[-1,-1,4,5,2,0],"fingers":[0,0,3,4,1,0],"fret":0},{"id":"0754xx","listCapos":[],"noteIndex":4,"notes":[-1,-1,47,43,40,28],"frets":[-1,-1,4,5,7,0],"fingers":[0,0,1,2,4,0],"fret":4},{"id":"01099xx","listCapos":[{"fret":9,"startString":2,"lastString":3,"finger":1}],"noteIndex":4,"notes":[-1,-1,52,47,43,28],"frets":[-1,-1,9,9,10,0],"fingers":[0,0,0,0,2,0],"fret":9},{"id":"010912xx","listCapos":[],"noteIndex":4,"notes":[-1,-1,55,47,43,28],"frets":[-1,-1,12,9,10,0],"fingers":[0,0,4,1,2,0],"fret":9},{"id":"121099xx","listCapos":[{"fret":9,"startString":2,"lastString":3,"finger":1}],"noteIndex":4,"notes":[-1,-1,52,47,43,40],"frets":[-1,-1,9,9,10,12],"fingers":[0,0,0,0,2,4],"fret":9},{"id":"x7x987","listCapos":[],"noteIndex":4,"notes":[59,55,52,-1,40,-1],"frets":[7,8,9,-1,7,-1],"fingers":[2,3,4,0,1,0],"fret":7},{"id":"0x200x","listCapos":[],"noteIndex":4,"notes":[-1,47,43,40,-1,28],"frets":[-1,0,0,2,-1,0],"fingers":[0,0,0,1,0,0],"fret":0}],"D":[{"id":"xx0232","listCapos":[],"noteIndex":2,"notes":[54,50,45,38,-1,-1],"frets":[2,3,2,0,-1,-1],"fingers":[2,3,1,0,0,0],"fret":0},{"id":"x54232","listCapos":[{"fret":2,"startString":0,"lastString":2,"finger":1}],"noteIndex":2,"notes":[54,50,45,42,38,-1],"frets":[2,3,2,4,5,-1],"fingers":[0,2,0,3,4,0],"fret":2},{"id":"xx0775","listCapos":[],"noteIndex":2,"notes":[57,54,50,38,-1,-1],"frets":[5,7,7,0,-1,-1],"fingers":[1,4,3,0,0,0],"fret":5},{"id":"x57775","listCapos":[{"fret":5,"startString":0,"lastString":4,"finger":1}],"noteIndex":2,"notes":[57,54,50,45,38,-1],"frets":[5,7,7,7,5,-1],"fingers":[0,4,3,2,0,0],"fret":5},{"id":"xx0111010","listCapos":[{"fret":10,"startString":0,"lastString":1,"finger":1}],"noteIndex":2,"notes":[62,57,54,38,-1,-1],"frets":[10,10,11,0,-1,-1],"fingers":[0,0,2,0,0,0],"fret":10},{"id":"101212111010","listCapos":[{"fret":10,"startString":0,"lastString":5,"finger":1}],"noteIndex":2,"notes":[62,57,54,50,45,38],"frets":[10,10,11,12,12,10],"fingers":[0,0,2,4,3,0],"fret":10},{"id":"xx12111010","listCapos":[{"fret":10,"startString":0,"lastString":1,"finger":1}],"noteIndex":2,"notes":[62,57,54,50,-1,-1],"frets":[10,10,11,12,-1,-1],"fingers":[0,0,2,3,0,0],"fret":10},{"id":"x5423x","listCapos":[],"noteIndex":2,"notes":[-1,50,45,42,38,-1],"frets":[-1,3,2,4,5,-1],"fingers":[0,2,1,3,4,0],"fret":2},{"id":"x5777x","listCapos":[{"fret":7,"startString":1,"lastString":3,"finger":3}],"noteIndex":2,"notes":[-1,54,50,45,38,-1],"frets":[-1,7,7,7,5,-1],"fingers":[0,0,0,0,1,0],"fret":5},{"id":"109777x","listCapos":[{"fret":7,"startString":1,"lastString":3,"finger":1}],"noteIndex":2,"notes":[-1,54,50,45,42,38],"frets":[-1,7,7,7,9,10],"fingers":[0,0,0,0,3,4],"fret":7},{"id":"1012121110x","listCapos":[{"fret":10,"startString":1,"lastString":5,"finger":1}],"noteIndex":2,"notes":[-1,57,54,50,45,38],"frets":[-1,10,11,12,12,10],"fingers":[0,0,2,4,3,0],"fret":10},{"id":"10977xx","listCapos":[{"fret":7,"startString":2,"lastString":3,"finger":1}],"noteIndex":2,"notes":[-1,-1,50,45,42,38],"frets":[-1,-1,7,7,9,10],"fingers":[0,0,0,0,3,4],"fret":7},{"id":"10121211xx","listCapos":[],"noteIndex":2,"notes":[-1,-1,54,50,45,38],"frets":[-1,-1,11,12,12,10],"fingers":[0,0,2,4,3,1],"fret":10},{"id":"x5x232","listCapos":[{"fret":2,"startString":0,"lastString":2,"finger":1}],"noteIndex":2,"notes":[54,50,45,-1,38,-1],"frets":[2,3,2,-1,5,-1],"fingers":[0,2,0,0,4,0],"fret":2},{"id":"10x777x","listCapos":[{"fret":7,"startString":1,"lastString":3,"finger":1}],"noteIndex":2,"notes":[-1,54,50,45,-1,38],"frets":[-1,7,7,7,-1,10],"fingers":[0,0,0,0,0,4],"fret":7},{"id":"10x121110x","listCapos":[],"noteIndex":2,"notes":[-1,57,54,50,-1,38],"frets":[-1,10,11,12,-1,10],"fingers":[0,2,3,4,0,1],"fret":10}]},"last_comments":[{"id":"4879208","rowid":"1429865","text":"Nice work!\r\nMaybe every second D could be an Am as well.... you can see it in the acoustic version. But D sound great too...","author_userid":"1881350","rating":"3","userid":"5758783","usergroupid":"2","username":"fameq2","date_created":"1411813214","type":"4","parent_id":0,"added_from":"","updated_at":null,"status":"new","moderator_user_id":"11953509","likes_count":"3","dislikes_count":"0","contributor_level":null,"avatar":"https:\/\/tabs.ultimate-guitar.com\/static\/users\/3\/8\/7\/5758783.gif?d=d","date":1411813214,"user_iq":10,"level":0},{"id":"5268351","rowid":"1429865","text":"its awesome i love it","author_userid":"1881350","rating":"0","userid":"12431037","usergroupid":"12","username":"jacobkilp99","date_created":"1504573328","type":"16","parent_id":0,"added_from":"","updated_at":null,"status":"new","moderator_user_id":"0","likes_count":"0","dislikes_count":"0","contributor_level":null,"avatar":"https:\/\/tabs.ultimate-guitar.com\/static\/users\/7\/3\/0\/12431037.gif?d=d","date":1504573328,"user_iq":10,"level":0},{"id":"5762607","rowid":"1429865","text":"Anyone, strumming pattern? :\/","author_userid":"1881350","rating":"0","userid":"19747408","usergroupid":"21","username":"tetroboylp","date_created":"1557406768","type":"4","parent_id":0,"added_from":"","updated_at":null,"status":"new","moderator_user_id":"0","likes_count":"0","dislikes_count":"0","contributor_level":"2","avatar":"https:\/\/tabs.ultimate-guitar.com\/static\/users\/8\/0\/4\/19747408.gif?d=d","date":1557406768,"user_iq":121,"level":0}],"comments":3,"stats":{"view_total":"29889","favorites_count":"1152"},"tab_corrects":[],"is_simplify_available":false,"can_edit_tab":false,"can_edit_strumming":false,"can_send_tab_stats":true,"official_backingtrack":null,"tab_search_link":"https:\/\/www.ultimate-guitar.com\/search.php?band_name=Radical+Face&song_name=the+mute&type%5B0%5D=200&type%5B1%5D=300","wiki_tab_user_group_id":21,"wiki_tab_user_iq":121,"user_group_id":21,"count_rating":73,"encode_strummings":"{\"patterns\":[{\"part\":\"Main Pattern\",\"denuminator\":16,\"bpm\":112,\"is_triplet\":0,\"measures\":[1,202,3,202,1,101,1,101]},{\"part\":\"Variation\",\"denuminator\":16,\"bpm\":112,\"is_triplet\":0,\"measures\":[1,202,3,101,1,202,3,101]}]}","alert":null,"recommendations":[{"id":1006040,"song_name":"Someone Like You","artist_name":"Adele","type":"Chords","part":"","version":2,"votes":2651,"rating":4.7495799999999999130295691429637372493743896484375,"date":"1290643201","status":"approved","preset_id":14669,"tab_access_type":"public","tp_version":0,"tonality_name":"A","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/adele_20519","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/adele\/someone_like_you_chords_1006040","date_update":"1552656730","user_id":611059,"user_iq":1694,"username":"watfordkev","type_name":"Chords","song_id":241854,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/adele\/someone_like_you_guitar_pro_1512038"},{"id":1171909,"song_name":"Radioactive","artist_name":"Imagine Dragons","type":"Chords","part":"","version":1,"votes":13466,"rating":4.87141000000000001790567694115452468395233154296875,"date":"1343779201","status":"approved","preset_id":2616,"tab_access_type":"public","tp_version":0,"tonality_name":"Bm","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/imagine_dragons_36784","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/imagine_dragons\/radioactive_chords_1171909","date_update":"1558571800","user_id":0,"user_iq":0,"username":"Unregistered","type_name":"Chords","song_id":313255,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/imagine_dragons\/radioactive_guitar_pro_2076405"},{"id":1706742,"song_name":"The Night We Met","artist_name":"Lord Huron","type":"Tabs","part":"","version":1,"votes":195,"rating":4.93137000000000025323743102489970624446868896484375,"date":"1422610276","status":"approved","preset_id":26539,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/lord_huron_32668","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/lord_huron\/the_night_we_met_tabs_1706742","date_update":"1485418878","user_id":8599286,"user_iq":172,"username":"Indrid_Cold","type_name":"Tab","song_id":1585910,"best_pro_tab_url":""},{"id":1707913,"song_name":"My Incredibly Nerdy Cute Little Lady","artist_name":"Dandelion Hands","type":"Chords","part":"","version":1,"votes":24,"rating":4.940889999999999560031938017345964908599853515625,"date":"1423212894","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/dandelion_hands_58923","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/dandelion_hands\/my_incredibly_nerdy_cute_little_lady_chords_1707913","date_update":"1485419063","user_id":3035273,"user_iq":195,"username":"nowaiii","type_name":"Chords","song_id":1586977,"best_pro_tab_url":""},{"id":1742169,"song_name":"Im Tired And I Hate This Song Dont Go","artist_name":"Cyberbully Mom Club","type":"Chords","part":"","version":1,"votes":34,"rating":4.7136300000000002086153472191654145717620849609375,"date":"1433327118","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":null,"verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/cyberbully_mom_club_58987","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/cyberbully_mom_club\/im_tired_and_i_hate_this_song_dont_go_chords_1742169","date_update":"1476176963","user_id":9356178,"user_iq":116,"username":"flamingomoo","type_name":"Chords","song_id":1606568,"best_pro_tab_url":""},{"id":1749724,"song_name":"Forest Fires","artist_name":"Axel Flóvent","type":"Chords","part":"","version":1,"votes":108,"rating":4.92257000000000033423930290155112743377685546875,"date":"1436438376","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":null,"verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/61579","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/1749724","date_update":"1469606013","user_id":4363451,"user_iq":62,"username":"halldor95k","type_name":"Chords","song_id":1611606,"best_pro_tab_url":""},{"id":1798571,"song_name":"Its Ok I Wouldnt Remember Me Either","artist_name":"Crywank","type":"Tabs","part":"","version":1,"votes":24,"rating":4.79270999999999958163243718445301055908203125,"date":"1451200838","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/crywank_47304","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/crywank\/its_ok_i_wouldnt_remember_me_either_tabs_1798571","date_update":"1518767024","user_id":6243851,"user_iq":52,"username":"sulaimanbutt1","type_name":"Tab","song_id":1639826,"best_pro_tab_url":""}],"official_backingtrack_url":null,"official_tab_url":null,"official_tab_want":{"artist_id":21222,"song_id":1419512,"vote_count":1},"tab_manually_highlighted":true,"pro_tab_onbording":null},"access":{"can_edit_tab":false,"can_edit_strumming":false,"can_delete_comment":false,"can_manage_rating":false,"can_vote_strumming":true},"recommended_tabs":[{"id":1006040,"song_name":"Someone Like You","artist_name":"Adele","type":"Chords","part":"","version":2,"votes":2651,"rating":4.7495799999999999130295691429637372493743896484375,"date":"1290643201","status":"approved","preset_id":14669,"tab_access_type":"public","tp_version":0,"tonality_name":"A","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/adele_20519","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/adele\/someone_like_you_chords_1006040","date_update":"1552656730","user_id":611059,"user_iq":1694,"username":"watfordkev","type_name":"Chords","song_id":241854,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/adele\/someone_like_you_guitar_pro_1512038"},{"id":1171909,"song_name":"Radioactive","artist_name":"Imagine Dragons","type":"Chords","part":"","version":1,"votes":13466,"rating":4.87141000000000001790567694115452468395233154296875,"date":"1343779201","status":"approved","preset_id":2616,"tab_access_type":"public","tp_version":0,"tonality_name":"Bm","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/imagine_dragons_36784","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/imagine_dragons\/radioactive_chords_1171909","date_update":"1558571800","user_id":0,"user_iq":0,"username":"Unregistered","type_name":"Chords","song_id":313255,"best_pro_tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/imagine_dragons\/radioactive_guitar_pro_2076405"},{"id":1706742,"song_name":"The Night We Met","artist_name":"Lord Huron","type":"Tabs","part":"","version":1,"votes":195,"rating":4.93137000000000025323743102489970624446868896484375,"date":"1422610276","status":"approved","preset_id":26539,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/lord_huron_32668","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/lord_huron\/the_night_we_met_tabs_1706742","date_update":"1485418878","user_id":8599286,"user_iq":172,"username":"Indrid_Cold","type_name":"Tab","song_id":1585910,"best_pro_tab_url":""},{"id":1707913,"song_name":"My Incredibly Nerdy Cute Little Lady","artist_name":"Dandelion Hands","type":"Chords","part":"","version":1,"votes":24,"rating":4.940889999999999560031938017345964908599853515625,"date":"1423212894","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/dandelion_hands_58923","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/dandelion_hands\/my_incredibly_nerdy_cute_little_lady_chords_1707913","date_update":"1485419063","user_id":3035273,"user_iq":195,"username":"nowaiii","type_name":"Chords","song_id":1586977,"best_pro_tab_url":""},{"id":1742169,"song_name":"Im Tired And I Hate This Song Dont Go","artist_name":"Cyberbully Mom Club","type":"Chords","part":"","version":1,"votes":34,"rating":4.7136300000000002086153472191654145717620849609375,"date":"1433327118","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":null,"verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/cyberbully_mom_club_58987","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/cyberbully_mom_club\/im_tired_and_i_hate_this_song_dont_go_chords_1742169","date_update":"1476176963","user_id":9356178,"user_iq":116,"username":"flamingomoo","type_name":"Chords","song_id":1606568,"best_pro_tab_url":""},{"id":1749724,"song_name":"Forest Fires","artist_name":"Axel Flóvent","type":"Chords","part":"","version":1,"votes":108,"rating":4.92257000000000033423930290155112743377685546875,"date":"1436438376","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":null,"verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/61579","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/1749724","date_update":"1469606013","user_id":4363451,"user_iq":62,"username":"halldor95k","type_name":"Chords","song_id":1611606,"best_pro_tab_url":""},{"id":1798571,"song_name":"Its Ok I Wouldnt Remember Me Either","artist_name":"Crywank","type":"Tabs","part":"","version":1,"votes":24,"rating":4.79270999999999958163243718445301055908203125,"date":"1451200838","status":"approved","preset_id":0,"tab_access_type":"public","tp_version":0,"tonality_name":"","version_description":"","verified":0,"recording":{"is_acoustic":0,"tonality_name":"","performance":null,"recording_artists":[]},"artist_url":"https:\/\/www.ultimate-guitar.com\/artist\/crywank_47304","tab_url":"https:\/\/tabs.ultimate-guitar.com\/tab\/crywank\/its_ok_i_wouldnt_remember_me_either_tabs_1798571","date_update":"1518767024","user_id":6243851,"user_iq":52,"username":"sulaimanbutt1","type_name":"Tab","song_id":1639826,"best_pro_tab_url":""}],"recentry_viewed_key":"tab:1429865","criticals":[]},"header_bidding":{"page_type":"tab_desktop","ad_units":[{"code":"ad_cs_2010320_728_90","dfpCode":"UG_ATF_728","sizes":[[728,90]],"placementId":"7478445","layoutPlacement":"top","appnexusPlacementId":12097756,"mediaType":"banner","bids":[{"bidder":"appnexus","params":{"placementId":7478445,"keywords":{"is_refresh":"false","artist":21222,"instr":"guitar","title":"Radical Face - The Mute (chords)"}}},{"bidder":"adform","params":{"mid":"434123"}},{"bidder":"aol","params":{"network":"9527.1","placement":"3211475"}},{"bidder":"brealtime","params":{"placementId":"11983206"}},{"bidder":"conversant","params":{"site_id":"110113","secure":"true"}},{"bidder":"districtM","params":{"placementId":"9325327"}},{"bidder":"indexExchange","params":{"id":"2","siteID":"179681","size":[728,90],"siteId":"179681"}},{"bidder":"mantis","params":{"property":"58c8252bdf390a2c0c1dba34","zoneId":"ug-us-728x90"}},{"bidder":"openx","params":{"delDomain":"ultimate-guitar-d.openx.net","unit":"538591653","customParams":{"variation":"340"}}},{"bidder":"pubmatic","params":{"publisherId":"72623","adSlot":"3131195@728x90"}},{"bidder":"pulsepoint","params":{"cf":"728x90","cp":"553934","ct":"484579"}},{"bidder":"rhythmone","params":{"placementId":"61405"}},{"bidder":"rubicon","params":{"accountId":"15178","siteId":"93918","zoneId":"441934"}},{"bidder":"sonobi","params":{"dom_id":"ad_cs_2010320_728_90","placement_id":"cdcb6dbf3c18a44c77a6"}},{"bidder":"improvedigital","params":{"placementId":"1123951"}},{"bidder":"oftmedia","params":{"placementId":"12698380"}},{"bidder":"onefiftytwo","params":{"networkId":"9969","siteId":"992300","zoneIds":[180104]}},{"bidder":"adkernelAdn","params":{"pubId":50158,"host":"tag.musiciansaudience.com"}},{"bidder":"sortable","params":{"tagId":"ug_desktop_728","siteId":"s2s.ultimate-guitar.com"}}],"variation":"340","variationName":"B"},{"code":"ad_cs_2010320_160_600","dfpCode":"UG_ATF_160","sizes":[[160,600]],"placementId":"7478445","layoutPlacement":"side","refresh":{"delay":30000,"delayDeviation":10000,"maxVisibleTime":1000,"limit":20},"appnexusPlacementId":12097756,"mediaType":"banner","bids":[{"bidder":"appnexus","params":{"placementId":7478445,"keywords":{"is_refresh":"false","artist":21222,"instr":"guitar","title":"Radical Face - The Mute (chords)"}}},{"bidder":"adform","params":{"mid":"434117"}},{"bidder":"aol","params":{"network":"9527.1","placement":"3211476"}},{"bidder":"brealtime","params":{"placementId":"11983208"}},{"bidder":"conversant","params":{"site_id":"110113","secure":"true"}},{"bidder":"districtM","params":{"placementId":"9325303"}},{"bidder":"indexExchange","params":{"id":"1","siteID":"177325","size":[160,600],"siteId":"177325"}},{"bidder":"mantis","params":{"property":"58c8252bdf390a2c0c1dba34","zoneId":"ug-us-160x600"}},{"bidder":"openx","params":{"delDomain":"ultimate-guitar-d.openx.net","unit":"538591659","customParams":{"variation":"340"}}},{"bidder":"pubmatic","params":{"publisherId":"72623","adSlot":"3131193@160x600"}},{"bidder":"pulsepoint","params":{"cf":"160x600","cp":"553934","ct":"484577"}},{"bidder":"rhythmone","params":{"placementId":"61405"}},{"bidder":"rubicon","params":{"accountId":"15178","siteId":"93918","zoneId":"441934"}},{"bidder":"sonobi","params":{"dom_id":"ad_cs_2010320_160_600","placement_id":"de32675870a56a0586a0"}},{"bidder":"improvedigital","params":{"placementId":"1123950"}},{"bidder":"oftmedia","params":{"placementId":"12698370"}},{"bidder":"onefiftytwo","params":{"networkId":"9969","siteId":"992300","zoneIds":[180103]}},{"bidder":"adkernelAdn","params":{"pubId":50158,"host":"tag.musiciansaudience.com"}},{"bidder":"sortable","params":{"tagId":"ug_desktop_160","siteId":"s2s.ultimate-guitar.com"}}],"variation":"340","variationName":"B"},{"code":"ad_cs_2010319_160_600","dfpCode":"UG_BTF_160","sizes":[[160,600]],"placementId":"7478445","dynamic":true,"layoutPlacement":"sideBottom","appnexusPlacementId":9493688,"mediaType":"banner","bids":[{"bidder":"appnexus","params":{"placementId":7478445,"keywords":{"is_refresh":"false","artist":21222,"instr":"guitar","title":"Radical Face - The Mute (chords)"}}},{"bidder":"improvedigital","params":{"placementId":"1123952"}},{"bidder":"oftmedia","params":{"placementId":"12698401"}},{"bidder":"onefiftytwo","params":{"networkId":"9969","siteId":"992300","zoneIds":[180105]}},{"bidder":"adkernelAdn","params":{"pubId":50158,"host":"tag.musiciansaudience.com"}}],"variation":"340","variationName":"B"},{"code":"ad_cs_2010319_300_250","dfpCode":"UG_BTFR_300","sizes":[[300,250],[610,300],[535,300],[535,400]],"placementId":"8549997","layoutPlacement":"bottom","refresh":{"delay":30000,"delayDeviation":10000,"maxVisibleTime":1000,"limit":20},"appnexusPlacementId":12097784,"mediaType":"banner","bids":[{"bidder":"appnexus","params":{"placementId":8549997,"keywords":{"is_refresh":"false","artist":21222,"instr":"guitar","title":"Radical Face - The Mute (chords)"}}},{"bidder":"adform","params":{"mid":"434119"}},{"bidder":"aol","params":{"network":"9527.1","placement":"4136371"}},{"bidder":"districtM","params":{"placementId":"9325371"}},{"bidder":"indexExchange","params":{"id":"4","siteID":"179683","size":[300,250],"siteId":"179683"}},{"bidder":"mantis","params":{"property":"58c8252bdf390a2c0c1dba34","zoneId":"ug-us-300x250"}},{"bidder":"openx","params":{"delDomain":"ultimate-guitar-d.openx.net","unit":"538591661","customParams":{"variation":"340"}}},{"bidder":"pubmatic","params":{"publisherId":"72623","adSlot":"3131198@300x250"}},{"bidder":"pulsepoint","params":{"cf":"300x250","cp":"553934","ct":"484580"}},{"bidder":"rhythmone","params":{"placementId":"61405"}},{"bidder":"rubicon","params":{"accountId":"15178","siteId":"93918","zoneId":"467536"}},{"bidder":"sonobi","params":{"dom_id":"ad_cs_2010319_300_250","placement_id":"2e4a8eab66953df9ce7a"}},{"bidder":"improvedigital","params":{"placementId":"1123953"}},{"bidder":"oftmedia","params":{"placementId":"12698411"}},{"bidder":"onefiftytwo","params":{"networkId":"9969","siteId":"992300","zoneIds":[180107]}},{"bidder":"adkernelAdn","params":{"pubId":50158,"host":"tag.musiciansaudience.com"}},{"bidder":"sortable","params":{"tagId":"ug_desktop_300R","siteId":"s2s.ultimate-guitar.com"}}],"variation":"340","variationName":"B"},{"code":"ad_cs_13538099_1_1","dfpCode":"UG_ATF_1x1","sizes":[[1,1]],"layoutPlacement":"offSide","refresh":{"enabled":false},"dynamic":true,"appnexusPlacementId":12097756,"mediaType":"banner","bids":[{"bidder":"justpremium","params":{"zone":"46762"}}],"variation":"340","variationName":"B"},{"code":"ad_cs_10000000_2_2","dfpCode":"UG_ATG_728_BX_MASTHEAD","sizes":[[1,1]],"placementId":"10000000","layoutPlacement":"outOfPage","refresh":{"enabled":false},"mediaType":"banner","bids":[],"variation":"340","variationName":"B"}]},"hasLesson":false};
This object turned out to contain ALL the contents of the page, ready for the other scripts to take the content and use it to populate the page itself! This discovery rapidly sped up my scraping and I was able to retake my scraper from the beginning, add some bodged-on functionality for multithreading (using a filesystem to syncronize multiple independent python processes, rather than actually have them communicate with something like mutexes and all that stuff), and make it scrape about 10+ pages per second!
10+ pages per second! This was a great step up from my previous attempt which averaged ~2 every second.
The final problem to work around was that after about an hour of repeated requests, they blocked my IP. Drat. All that euphoria of finally having a functional and moderately efficient scraper, and now this.
The solution was to employ a CLI-based VPP which Windscribe offers, and have a script that restarted the VPN every ten minutes or so. Perhaps this wasn't the best solution, but it worked for my purposes and I only had to use it for about a day.
You can check out the app that the scraping facilitated here, and a blog post that demonstrates how it works here. I'm not posting the code itself, because other scrapers of this website have been removed in the past, but for anyone looking to do something similar, I hope this was mildly informative!