After seeing some relative success in our Solr implementations xml response times by switching on Tomcats http gzip compression, I've been doing some comparisons between the other formats solr can return. We use Solrnet, an excellent open source .NET Solr client. At the moment, it only supports xml responses, but every request sends the "Accept-encoding:gzip" header as standard, so all you have to do is switch it on on your server and you've got some nicely compressed responses. There is talk of supporting javabin de-serialisation, but it's not there yet. I've decided to compare the following using curl with 1000 rows and 10000 rows in json, javabin, json/gzip compressed and javabin/gzip compressed.

My test setup is a solr 1.4 instance with around 11000 records in sitting behind an nginx reverse proxy handling the gzip compression. As I said, this could easily be achieved by switching on gzip compression in Apache Tomcat. The same 10000 records, returned using the q=*:* directive with wt=json when http gzip compressed is the smallest, but only marginally, compared to wt=javabin. It would seem that json compresses very well indeed. You can also see the massive drop just switching on gzip compression gives to xml.

My conclusion to this would be that because json is a widely accepted content-type, with many well known and fast de-serialising libraries, it would probably be worth implementing that rather than trying to de-serialise javabin. But this was only a quick test and does't take into account how quickly solr handles serialisation of the documents server-side.

Tag: 
Solr
Anonymous
Friday, July 17, 2015 - 20:18

by Alan Hannaway, 7digital Product Owner for Data 

We often ask ourselves How different do you think our listening experience will be in the next ten years? It’s a difficult question to answer, but a great one to ask. Serving an industry where there is constant change, the question brings us right back to where we should be focused: the way people experience music and radio.

Having powered music and radio services for over 10 years, 7digital knows how to deliver listening experiences that delight millions of people. We regularly reflect on what works, and what doesn’t. Sometimes it is clear what works well, and if you have a culture where you fail early and loudly (we do; it is part of our tech manifesto) you can sometimes see exactly what you did wrong. It’s not always easy though, and when the reason for something happening is not at all clear, finding out why it happened is difficult. How can you make sure the reasons you say something happened, are because of the reason you have identified? Correlation does not imply causation.

sharri.morris@7digital.com
Thursday, June 25, 2015 - 15:14

sharri.morris@7digital.com
Wednesday, May 27, 2015 - 17:21

We have updated the SolrNet repository so that the Structuremap implementation now allows for multi-core instances. Which can be set up in your Bootstrapper like this:

var solr = (SolrConfigurationSection)ConfigurationManager.GetSection(“solr”); var solrServers = solr.SolrServers; ObjectFactory.Initialize( x => x.AddRegistry(new SolrNetRegistry(solrServers)) );

 

You're app config should look like the following:

hack 24 logo
admin
Thursday, April 30, 2015 - 15:56

This Spring, on the 2nd and 3rd of May, 7digital are proudly sponsoring a new hack day, set up by the Tech Nottingham community - Hack24.

During this 24 hour coding competition, teams of 4 compete to win exclusive prizes, by showcasing their innovative ideas and raw tech talent!

Set in Nottingham's Creative Quarter, this will definitely be an exciting Hackathon to be a part of.

How to get involved?