After seeing some relative success in our Solr implementations xml response times by switching on Tomcats http gzip compression, I've been doing some comparisons between the other formats solr can return. We use Solrnet, an excellent open source .NET Solr client. At the moment, it only supports xml responses, but every request sends the "Accept-encoding:gzip" header as standard, so all you have to do is switch it on on your server and you've got some nicely compressed responses. There is talk of supporting javabin de-serialisation, but it's not there yet. I've decided to compare the following using curl with 1000 rows and 10000 rows in json, javabin, json/gzip compressed and javabin/gzip compressed.

My test setup is a solr 1.4 instance with around 11000 records in sitting behind an nginx reverse proxy handling the gzip compression. As I said, this could easily be achieved by switching on gzip compression in Apache Tomcat. The same 10000 records, returned using the q=*:* directive with wt=json when http gzip compressed is the smallest, but only marginally, compared to wt=javabin. It would seem that json compresses very well indeed. You can also see the massive drop just switching on gzip compression gives to xml.

My conclusion to this would be that because json is a widely accepted content-type, with many well known and fast de-serialising libraries, it would probably be worth implementing that rather than trying to de-serialise javabin. But this was only a quick test and does't take into account how quickly solr handles serialisation of the documents server-side.

Tag: 
Solr
sharri.morris@7digital.com
Tuesday, April 28, 2015 - 16:27

Why metrics? Since I joined 7digital I've seen the API grow from a brand new feature side by side with the (then abundant) websites to be the main focus of the company. The traffic grew and grew and keeps on growing in an accelerated pace and that brings us new challenges. We've brought the agile perspective into play which has made us adapt faster and make fewer errors but:

sharri.morris@7digital.com
Tuesday, March 17, 2015 - 12:49

I wanted to start looking at alternatives to our current set of cucumber feature tests. At the moment on the web team we're using using FireWatir and Capybara. So I though I'd take at look at what was available in Node.js. Many people think it's strange that a .Net shop would use a something written for testing Ruby or even consider something that isn't from the .Net community. Personally I think it's a benefit to truly look at something form the outside in.  Should it matter what you're using to drive your end product or what language your using to test it? Not really. So what are the motivations for moving away from Ruby, Capybara and FireWatir? In a word 'flaky', we've had heaps of issues getting our feature tests, AATs and smoke tests reliable. When it comes to testing, consistency should be king. They should be as solid as your unit tests.  If they fail you want to know that for definite you've broken something, rather than thinking it's a problem with the webdriver. It is with this aim in mind that I started looking at the following. Cucumber.js is definitely in it's infancy, there's lots of stuff missing but there's enough there to get going. Zombie.js is a headless browser, it claims to be insanely fast, no complaints here.

anna.siegel@7digital.com
Wednesday, March 11, 2015 - 12:27

 

Following changes to our Catalogue API, we are releasing a change to the Basket API to support premium quality formats.

This release adds a package element below basketItem in all basket responses. This is to support the sale of music in different audio formats.

An example response would now look like this:

Anonymous
Tuesday, February 3, 2015 - 03:08

Guest Blog by John Nye on His First Week at 7digital

I have recently started at 7digital and already there are a few things of note that may seem small, but highlight the differences in attitude 7digital take over other companies I have worked for. Below are a few thoughts from my first week at 7digital.

Day 1 - Meet the team

After an incredibly frustrating start, owing to a 4 hour delay on my train journey, I was introduced to everyone, given my security pass and directed to a new starter guide that had a series of tasks that needed to be completed. These tasks ranged from installing software to getting added to email groups to reading up on the 7digital handbooks. So I logged into my Ubuntu machine and started going through the list... wait... Ubuntu?

Thoughts from the day:

  • Incredibly welcoming bunch.
  • Locate an Ubuntu book!

Day 2 - Empowerment