Game Changing Technology

NoSQL expands the art of the possible with public sector data

Thinking innovatively: Scenario 1: It’s a scene all levels of government can relate to — volumes of data scattered in databases and file systems in different formats, unable to be accessed, searched or analysed easily. To more quickly respond to property history requests, Local Government developed a repository for land use data residing in a MarkLogic NoSQL database where it can be viewed, correlated and analysed. Tasks that once took days or weeks now take minutes or even seconds.

Thinking innovatively: Scenario 2: The Defence Department created the Distributed Common Ground System (DCGS) to enable all branches of the DOD, including

the military, to share information via the cloud. But with different agencies and military services using different data collection and storage systems based on relational database technology, searching and sharing is complex and laborious. An independent test consisting of 20 million records using MarkLogic Server showed that the same amount of work could be done 59 times faster, with average response time 32 times faster than the previous system.

Thinking innovatively: Scenario 3: The Federal Aviation Administration needed a way to make faster, more accurate decisions during crises. Built on the MarkLogic platform, the FAA’s Emergency Operations Network Dashboard combines all data sources including valuable semi- and unstructured
data sources such as PDFs, email, social media posts, weather reports and news stories. The system includes powerful search, discovery and collaboration, creating a common operating picture that allows personnel to properly manage emergencies on the ground and in the air.

While these Departments might have achieved similar results with existing technology, it would have taken much more time and money, and involved multiple databases. Faced with growing volumes of both traditional structured and newer unstructured data — social media, text, geographic data and much more — along with mandates to increase efficiency share information and improve service to the public, departments are beginning to face the fact that in many cases, traditional approaches and tools just aren’t measuring up.

But it’s even more than just becoming efficient; it’s about tapping the unrealised value in government databases by combining structured data in SQL-based databases with new sources of unstructured data that make better use of technologies more suited to handle them, like NoSQL and Hadoop.

“Agencies no longer have to limit their hypotheses to traditional data sources, but can expand their world to look at other data sources,” said Kevin Shelly, Vice President of MarkLogic Public Sector. By finding patterns that you haven’t been able to find before or took too long to find, agencies can take the quick, decisive action they need to take to save lives, find and mitigate risk, improve efficiency, save money and improve service to the public.”

MarkLogic is a database platform with a difference. As the only Enterprise NoSQL database, it combines billions of data points and documents from many highly disparate data sources into a secure, searchable repository. Not only does this create a single view of data, but also provides rich search, query and real- time alerting capabilities. And it’s lightning-fast: it can search and analyse petabyte scale data with sub-second response times.

It’s also extremely cost-effective. With systems built on MarkLogic, government departments can use less expensive distributed commodity hardware. That cost savings, combined with information ingestion that is 120 times faster than relational databases, improves performance on unstructured information by as much as 100 times (versus relational databases). It also reduces the amount of software code by 70 percent, decreases development time for new software applications by 50 percent, and reduces overall development costs by 90 percent.

By combining out-of-the-box thinking with next-generation technology, the public sector can achieve and surpass their goals and know that they are prepared for whatever is required.

For more information please visit