2 February 2017

Utilising the GoodReads API on BookFetch

The biggest problem revealed after Saturdays test of the BookFetch scanner was the systems inability to lookup 60% of the scanned barcodes on the ISBNdb.com API.

Owen, a friend in the local Bristol tech community suggested giving GoodReads.com's API a try. I added a fallback in the code that processes the scans so that if it should fail to obtain a result from ISBNdb.com's API, it then tries GoodReads.com's. The result was that the queue of 235 unprocessed codes reduced to just 29.

With the queue now much smaller it was easy to spot that 15 of them were clearly not valid ISBN numbers. To solve this problem I added some additional validation to the BookFetch Laravel API so that it rejects invalid scans by responding with a 422 Unprocessable Entity.

$messages = [ 
    'isbn.regex' => 'The :attribute field is not a valid ISBN number' 

$validator = Validator::make( $request->all(), [
    'isbn' => ['regex:/^97(8|9)[\d]*$/'],
    'shop_code' => 'required'
], $messages);

if( $validator->fails() ){
    return response( $validator->errors()->all(), 422 );

So what about the remaining 13 barcodes in the queue that are apparently valid ISBN numbers but did not yield a result from either ISBNdb or GoodReads? Well with a bit of manual checking I found that 12 of them could be found on Amazon's website. As far as I am aware Amazon's API is not free although I haven't actually researched the exact cost yet. If it's only 12 out of 400 scans that need looking up it might be worth considering a manual data input task. Anyway, that's for another time.

There is one final question of morality raised from this... If there are 3 competing APIs who all impose a limit on free usage. Is it cheating to spread your requests evenly across them in order to avoid incurring a cost?  The terms of service for each API state that you must credit the source of the information where it is displayed on your site but they do not go so far as to say it cannot appear alongside data retrieved from competing services. An interesting thought but again, a question to be saved for another time.

Finally, if you are interested in what the 1 remaining unprocessable number is, it is 9780434037209.