Posts Tagged ‘software’

Open eGovernment Data

Monday, November 28th, 2011

The Open Source movement is moving into government data.  Governments are finding a new source of untapped economic stimulus with the mountains of data they collect.  The  data is collected for the ultimate good of the public but rarely shared because information access was too people intensive and expensive up until recently.  Things have changed.

GOV opendata1 300x168 Open eGovernment Data

ETALAB (France), data.gov.uk (UK), data.dc.gov (Washington, DC, US), whitehouse.gov/open (US), and countless other local and national governments have open their data coffers.  In the case of DC for instance, the cost of publishing the data was $50K for the city. The DC government expected it to spur the creation of a few new ventures, and a bit of private investments.  Instead, 50 startups were born and $3M invested.  There is a world of open data coming to the private software industry.

Open Government data is also going to be Big Data.  The size of data collected is by definition larger larger than traditional “enterprise data” for instance (especially at the national level).  The tools being developed for big data will solve some of the issues with access and real time analytics that exit with government data.  Exorbyte MatchMaker is one of these tools.  That’s why government agencies have already chosen MatchMaker for their search and data access challenges (2 national European census agencies, German Finance Ministry, and more).

Are you ready for open government data?  Any ideas what would make sense to build with this data?

Big Data Search

Monday, November 28th, 2011

Big Data1 300x225 Big Data SearchEvery economic cycle comes with its host of enterprise software trends.  Big Data hs become a recognized phenomenon in 2011.  In May 2011 McKinsey released the “Big data: The next frontier for innovation, competition, and productivity” report. It started with:  ”The amount of data in our world has been exploding and analyzing large data sets—so-called big data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus”.

IBM, Oracle, SAP, Microsoft, SalesForce.com, and others are all aiming their development efforts at Big Data (see vldb.org).  The amount of data produced, collected and stored by online activities to which companies, their customers, their partners, and their sales channels participate has grown enormously.  Tools are being developed that allow affordable long-term storage.  New columnar in-memory database formats have emerged that enable near-real-time analytics.  Fast growing stratups and open source solutions have also converged with their own new NoSQL formats (InfiniDB, LucidDB, InfoBright, Hadoop, NoSQL, etc.)     MatchMaker, Exoryte’s Universal Search platform, is the perfect answer to search within Big Data.

The challenges of search within big data are:

  • Searching Big Data though SQL queries is simply too slow and inflexible – fuzzy or advanced search requires a search indexer layer or  something different than traditional on-disk relational DB formats.
  • Indexing large databases can be long, disruptive to normal database operation and require complex hardware infrastructures.
  • Running complex queries and fuzzy logic requires so much calculation and lookups that new search strategies are required.

Exorbyte MatchMaker is made to address these challenges and our professional services team has proven repeatedly tht they can be addressed:  Allianz (the world’s 12th-largest financial services group),  German Finance Ministry, and more blue chip and government organizations tun o us each year for that very expertise.

What do you think of Big Data?

Oracle Eying Endeca? (Oct 18th Update: Oracle Acquires Endeca)

Friday, September 30th, 2011

Oracle Larry Ellison has been attacking the rationale of HP buying Autonomy for $10B.  One wonders if he wasn’t eyeing Autonomy himself for an acquisition before HP snagged it.  This said, now that Fast, Convera, Autonomy, Netrics, have all been acquired, we are wondering if Endeca might not be the next one up for an Oracle acquisition this time?

See more here:

http://www.businessinsider.com/oracle-hp-autonomy-2011-9

http://www.marketwatch.com/story/oracle-fuels-fire-over-h-ps-autonomy-deal-2011-09-29

http://articles.businessinsider.com/2011-09-29/tech/30216980_1_oracle-qatalyst-slides

==== 10/18/2011 – Update ====

http://www.forbes.com/sites/ericsavitz/2011/10/18/oracle-buys-endeca-targeting-unstructured-data/ That’s it.  Oracle announces they are buying Endeca.  This is big news.  Another confirmation for us that Search is a hot category for enterprise software but also that deeper integration of search is in the works.  While this acquisition of may be a good thing for those Endeca users with full-text and unstructured  data applications, other with massive structured data (ecommerce, CRM, etc.) will need to turn to other solution providers possibly.  We are ready and waiting.

Low Hanging Fruit for Most Online Stores: Better Search Features

Wednesday, August 25th, 2010

We just launched the public beta version of Exorbyte Commerce Search and Autocomplete (if you are in the UK: http://commerce.exorbyte.co.uk). It’s a hosted SaaS monthly subscription version of our popular search products (MatchMaker and SearchNavigator) especially designed for ecommerce.

We also serve a large selection of other industries with the same core products for search within structured data. However we chose ecommerce, and we did so on purpose. There is plenty of data showing how many online stores are behind in terms of satisfying users with better search features to help them find what they want to buy. Here are  some of the reasons why Exorbyte Commerce is worth many times the very short time  (only minutes) setting up its free trial requires:

  • More than 47% of online shop visitors don’t become buyers because of poor search or navigation features.
    “According to iPerceptions’ E-commerce Industry Report Q2 2009, 38.6% of the 360,000 or so visitors to the 160 websites tracked were in the research stage of the buying process, while just 17% were at the buying stage.

    online shopper flight1 300x177 Low Hanging Fruit for Most Online Stores: Better Search Features
    source

    Source: iPerceptions' E-commerce Industry Report Q2 2009

    iPerceptions was able to establish several reasons why visitors aren’t purchasing. The main reason was that visitors weren’t able to find what they were looking for (34%), while price and navigation/usability issues tied in second place (13%) followed by shipping policies (9%).

    why customers dont buy1 300x200 Low Hanging Fruit for Most Online Stores: Better Search Features
    source

    Source: iPerceptions' E-commerce Industry Report Q2 2009

  • In his book The Humane Interface, renowned user interface guru Jef Raskin compares incremental and delimited search: “With a delimited search, the computer waits for the user to type a pattern and delimit it, after which it is the user who waits while the computer does the search. When using a delimited search the user must guess, beforehand, how much of a pattern the computer needs to distinguish the desired target from other, similar targets. With an incremental search, he can tell when he has typed enough to disambiguate the desired instance, because the target appeared on the display.”
    Jef Raskin goes on to say: “In spite of near agreement about the desirability of incremental searches on the part of both designers and users, almost all interface-building tools make it easy to implement delimited searches and difficult or impossible to implement incremental searches.”
    Jef Raskin even ventured to say in a footnote that search is either “incremental or excremental”. We kind of agree with him. icon wink Low Hanging Fruit for Most Online Stores: Better Search Features

While ecommerce catalogs are structured database tables, many online stores still use lame full-text search engines or slow database queries to provide catalog search results to their users. This is a real problem for the following reasons:

  • Full-text search engines usually have to crawl pages to index the data, making the update mechanism of search results slow and often out-of-date (not reflecting frequent changes in inventory levels, prices, and descriptions retail database undergo daily).
  • Database queries and full-text queries are simply not capable of handling advanced fuzzy search like structured data or database indexing engines can. Too slow or simply impossible to implement.
    Just try to build an advanced multi-field fuzzy search facility for a database of millions of SKUs (ex: online travel store, industrial parts, online electronics store, etc.) and keep it fast (under 10ms round trip) using a database query or full text search engine! Have fun and let us know if you want tips on how to make it work with a different approach.

Something that would be much better, much simpler, much faster and much more natural. An ecommerce search system that yeilds more conversions. Simply a system that guesses what you want to find even if you don’t know what it’s called: That is what we strive for every day at Exorbyte! What do you think?

Exorbyte Technology and Features Detailed, but Never Enough for Some

Monday, August 23rd, 2010

We are often asked by online requests from prospects and from potential users to give them extensive details on our data matching technology.  That’s understandable.  A few of the claims Exorbyte makes regarding the speed and error-tolerance of our software are pretty unique and attract interest by users and developers alike.  This said, it’s often with regret that we give partial information but cannot divulge the more proprietary parts of our system’s capabilities.

I would like to set the record straight.  Like all software companies, we have to retain a degree of privacy around the most valuable of our inventions.  However, we often see developers come to us for information and then attempt to build an application rivaling MatchMaker in speed and error tolerance.  The results are almost always disappointing.  The reason is simple:  there is no silver bullet.  

There is no single invention or trick to how MatchMaker can return Levenshtein or Soundex query results on many millions of data records in under 10 ms.  The performance our software features comes from a large series of architectural, indexing and algorithmic innovations.  Furthermore, every build of our core MatchMaker platform is tested for speed and overall performance at night (after our developers go home) to maintain optimal performance no matter what fancy features we add to the system.  This has been going on for 10 years now, so the quality, speed, depth of functionality, flexibility, and reliability of MatchMaker are no miracle.  We work at it every day.

We just released a new portion of our web site which describes there characteristics in more detail called Exorbyte Structured Data Search and Data Matching Platform.  Check it out and tell us what’s missing if anything.