Master Data Management – Five Things to Know

In the world of master data management, silos are a tremendous challenge.

When enterprises try to process information from disparate systems, they too often use sub-optimal applications and initiatives laden with errors and misinformation, not to mention blown timelines and budgets. But master data management (MDM) is actually more than just the breaking down of data silos. It’s about efficiency and service, innovation and security, clarity and perspective. It’s about getting the most of your most valuable resource: your data.

Here are the five things you need to know about MDM:

The Challenge of Multiple Data Systems

For existing enterprises, one of the largest hurdles to developing an MDM system is the multiplicity of databases and applications usually involved. What’s more, Enterprise Resource Planning and Customer Relationship Management systems rely on structured data, whereas the proliferation of IoT devices has created exponential growth in unstructured data.

Take the example of Enel, which is one of the largest power utilities in Europe. Enel was struggling to provide analytics and reporting across all of its power generation plants and equipment. They see data flowing in from multiple systems, including IoT devices on power generation equipment, plant maintenance systems, scheduling applications, and other sources. Each of these data sources has its own data types. Enel was exporting data to csv files and manually aligning the data to generate reports and analytics.

Other companies in similar scenarios might invest in expensive integration bus systems to support a polyglot persistent environment.

Enel found a solution in a native multi-model database. This allowed them to bring all data into a single database. This means no more worrying about different data types or keeping the different systems in sync. The result was real-time data analysis across all sites and multiple data systems. No more month-long manual processes to manually generate reports.

Master Data Management Really is for Everyone

All companies are now digital enterprises. Since all systems rely on data, MDM is a discipline in which all organizations need to remain competitive. Master data powers everything from financial reporting to real estate transactions to fraud protection. The ultimate results are faster and better decisions, improved customer satisfaction, enhanced operational efficiency, and a better bottom line.

Redundancy Elimination is Only Part of It

Most people who’ve heard of MDM immediately link it to one of its primary objectives: the elimination of redundant data. Yes—having a central repository of data will eliminate data redundancies, as long as it’s done correctly. But the benefits of MDM extend beyond redundancy elimination. Namely: data consistency, data flexibility, data usability, and data security (from role-based access).

Mergers and Acquisitions Don’t Have to Mean a Master Data Management Nightmare

Mergers and acquisitions can be rough on data consistency. Reconciling several master data systems brings headaches from different data system taxonomy and structures. This usually results in two systems remaining separate and linked only through a special reconciliation process.

As more acquisitions and mergers occur, the problem compiles into a labyrinth of siloed systems and data. This brings you back the problem that spurned you to invest in MDM in the first place.

The answer lies in the database management system and vendor you choose for your master data MDM system. Make sure to choose a vendor that offers a flexible, multi-model database that allows you to easily develop a single data taxonomy.

The Database that Backs Your Master Data Management System is Key

The most powerful and effective MDM systems run on databases that fit the business model in question.

As an example, Xima Software uses networks that are like graphs. As such, for a telco, an MDM system via a multi-model graph database is the most effective MDM strategy because the database allows for easy visualization of the network since it uses the same graph model.

Master Data Management is Evolving

If there were a fifth thing you needed to know about MDM, it’s that it’s rapidly evolving to meet the needs of today’s enterprises and their customers. Retailers are using it to improve time-to-market and address their customers’ growing expectations to deliver a true omnichannel experience. The consumer packaged goods industry is using it to ensure the accuracy of nutritional information and comply with local disclosure regulations. And every industry is using it to break down data silos.

Gerard (Jerry) Grassi, P.E.
Senior Vice President – OrientDB
SAP

London – June 1, 2017

The OrientDB Team has just released OrientDB v2.2.21

EE-2221

If you are using 2.2.x series, please upgrade your production environments to v2.2.21. View our Change Log for a full list of new functionalities, bug fixes and other improvements.

Download The Latest OrientDB (v2.2.21) version:

https://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: https://orientdb.com/download-previous/

Noteworthy Enhancements

For a complete list of enhancements, improvements and bug fixes, have a look at the complete OrientDB Changelog.

CORE

– Fixed bug on file management after restoring a 2.1.x backup on 2.2.x

STUDIO

– Fixed bug on client blocked in case of error on message reading

GRAPH

– Fixed integration with gremlin for result set of collectionsinfoicon

HIGH AVAILABILITY

– Improved performance of messages exchange
– Fixed locking problem with distributed transactions
– Fixed OSGi dependency of distributed module from graphdb

Team and Contributors

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

In case you missed some of the latest news and haven’t subscribed to our newsletter below, here’s a quick recap of this month’s news. Find out how OrientDB can help your startup company with our latest case study and how companies across the world are using graph technology to secure their systems. Stay up-to date with our latest stable release or test the new features in our 3.0 milestones release.

Case Studies

newsc_logo_bl@4xIf you want to learn about how OrientDB can help your startup company, take a look at our latest case studies. Find out how New.sc uses graphs and multi-model features to power an intuitive and increasingly popular news platform.


OrientDB Releases

OrientDB LabsEarlier this month we released OrientDB 2.2.20. This is our latest stable release so if you haven’t upgraded yet, go ahead and download it now. In case you missed it, last month we released our OrientDB 3.0 Milestones Edition. Though not yet suitable for production environments, if you want to test the latest features included in our upcoming 3.0 release, head over to our Labs page.


Partnerships

mi-me-comThis May OrientDB announced their partnership with Chinese system integrator and fraud detection experts MiMe. Using multi-model databases, MiMe is helping companies across China move from antiquated relational systems to modern day, innovative database systems.


Transforming Relational Data

With the release of OrientDB Teleporter last year, OrientDB is being used around the world to synchronise relational data. In fact, we’re the first NoSQL database to enable this feature. Whether it’s data from Oracle, SQLServer, MySQL, PostgreSQL or HyperSQL, Teleporter transforms tables to graphs and allows Relational and NoSQL technologies to coexist.


Stay tuned for more news,

The OrientDB Team


London – April 18, 2017

The OrientDB Team has just released OrientDB v3.0.0 M1

It is with great pleasure that we announce the release of the highly anticipated OrientDB 3.0! Improving upon the foundations established in OrientDB 2.2, this latest release further consolidates all our strengths to establish a new standard for Multi-model Databases.  As with all our releases, security is always of the utmost importance. OrientDB 3.0 solidifies all our security elements but also adds unprecedented speed and performance, perfect for virtually any use case.

A huge thanks goes out to our dedicated world wide community for their incredible support, relentless efforts and invaluable contributions.

Download The Latest OrientDB (v3.0.0 M1) version:

If you would like to give our latest OrientDB 3.0.0 Milestones Community Edition a try, head over to our OrientDB Labs page and download your copy. Remember that this release is for testing purposes only and is not Suitable for production environments.

Noteworthy Enhancements

We’ve outlined some of the main features of our latest edition in our OrientDB Labs page. For a complete list of features, have a look at the OrientDB 3.0 Release Notes.

 

Best Regards,

 

luca signature
Luca Garulli
CEO & Founder
OrientDB LTD

OrientDB Community Awards

OrientDB Community Awards

 

We value and appreciate the hard work put in by the world-wide OrientDB community. That’s why, as a small token of appreciation, we’ve started sending out some gadgets and rewards to our community members.  

 


Code Contributors

Stabrizi
Saeed Tabrizi

A special thank you to Saeed for his dedication to OrientDB. Among the numerous and valuable contributions, some noteworthy examples include Pull Requests on OrientJS repository in which, among several improvements, he implemented the IF NOT EXIST clause when creating classes and properties, and IF EXIST clause when dropping classes and properties.

mpollmeier
Michael Pollmeier

Michael is the original Author of the Apache TinkerPop 3 Graph Structure Implementation for OrientDB, which will be officially supported in upcoming major OrientDB releases!

 


Community Contributor

smolinari
Scott Molinari

Not only has Scott provided detailed bug reports and documents, he’s helped countless community members by shedding light on new features and helped countless others experiencing issues.

 


Thank You for Your Contributions

Thank you to Saeed, Michael and Scott, who as a gesture of appreciation will be receiving a Raspberry Pi 3® Starter Kit along with some OrientDB merchandise (T-shirt, stickers and that kind of stuff)**.

r-pi2orientdbshirts1

Next time – Bloggers and Writers

We’d also like to send out a special thank you to all the community members writing about OrientDB in their blogs, articles & papers. Thats why next time around we’ll be sending out some more gadgets to our top community bloggers.

So if you’re currently writing about @OrientDB, remember to use the the #OrientDB and #Multimodel tags in your posts and head back to this page regularly. You might find your name on our Top Contributors list!

*All trademarks are the property of their respective owners.
**All OrientDB Community Award winners will be contacted individually in order to receive their prize.

 

London – November 15, 2016

EE 22131

The OrientDB Team has just released OrientDB v2.2.13, resolving 34 issues from v2.2.12.

 

If you are using 2.2.x series, please upgrade your production environments to v2.2.13. For more information, take a look at the Change Log.

Download The latest OrientDB (v2.2.13) version:

https://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: https://orientdb.com/download-previous/

Noteworthy Enhacements

We are happy to announce that in this release we closed a long wanted enhancement (Issue #3597). IF NOT EXISTS clause are now available for the following commands:

As always, thanks to our community for their suggestions and patience. We welcome your feedback so if you have any enhancements for our products, please open an issue on GitHub.

Team and Contributors

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

London, June 20, 2016

The OrientDB Team has just released OrientDB v2.2.3, resolving 7 issues from v2.2.2. If you are still using 2.2.x (or 2.1 series), please upgrade your production environments to v2.2.3. For more information, take a look at the Change Log.

Download OrientDB v2.2.3 now: https://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: https://orientdb.com/download-previous/

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

OrientDB launches its Open Source NoSQL Graph-Document Database through CenturyLink’s Cloud Marketplace

 

LONDON, UK – March 17, 2016 OrientDB, the pioneer behind the world’s first Open Source, NoSQL distributed graph-document database, today announced its certification under the CenturyLink Cloud Marketplace Provider Program. Through this partnership, CenturyLink Cloud users are now able to deploy and manage OrientDB’s community or enterprise edition databases via CenturyLink’s Blueprints library.

OrientDB is a second-generation distributed graph database with the flexibility of documents and an open source Apache 2 license. By treating every vertex and edge as a JSON (JavaScript Object Notation) document, OrientDB enables the creation of multi-directional property graphs, allowing bulks of data to be traversed with ease. This new multi-model approach, with a polyglot engine, eliminates the need for multiple systems, ensures data consistency and optimizes the formation of complex relationships. Even for a document-based database, the relationships are managed, as in graph databases, with direct connections amongst records. Its versatility and rapid integration makes OrientDB a perfect candidate for use cases ranging from recommendation engines and fraud detection to real-time analytics and content management. Fortune 500 companies, government entities and startups all use the technology to build large-scale innovative applications.

CenturyLink Cloud customers can now benefit from OrientDB’s features:

OrientDB Community Edition is free for any purpose (including commercial use). OrientDB Enterprise Edition serves as an extension of the Community Edition by providing enterprise-needed features such as: Query Profiler, distributed clustering configuration, metrics recording, and Live Monitor with configurable alerts.

The CenturyLink Cloud Marketplace Provider Program allows participating technology companies, like OrientDB, to integrate with the CenturyLink Cloud platform. These additional business-ready solutions are available to CenturyLink’s cloud, hosting and network customers.

“Companies hoping to leverage big data are getting tired of dealing with multiple systems and increasing infrastructural costs,” said Luca Garulli, CEO of OrientDB. “Customers choose OrientDB for its innovative Multi-model database capabilities and affordable nature. Expanding our capabilities to the cloud through CenturyLink provides the perfect accessible solution without the need for multiple database systems or costly servers.”

“The foundation of the big data revolution on our platform has been software innovation around unstructured data management,” said David Shacochis, vice president of platform enablement at CenturyLink, “OrientDB is a great example of this trend, allowing our customers to manage their unstructured data relations in a scalable model that drives insight out of their business workloads”

To start using OrientDB on CenturyLink Cloud today, refer to the “Getting Started” guide on the CenturyLink Cloud Knowledge Base.

About OrientDB

OrientDB is an open source 2nd Generation Distributed Graph Database with the flexibility of Documents and a familiar SQL dialect. With downloads exceeding 70,000 per month, more than 100 community contributors and 1000’s of production users, OrientDB is experiencing tremendous growth in both community and Enterprise adoption. First generation Graph Databases lack the features that Big Data demands: multi-master replication, sharding and more flexibility for modern complex use cases. See for yourself, Download OrientDB and give it a try.

Editorial Contacts:
Paolo Puccini
OrientDB Ltd
+44 203 3971 609
info@orientdb.com

OrientDB v2.2 Beta Going Live Next Week!

 

We’ve worked hard on it,  it’s about to go live, we can’t contain our excitement, and of course, we couldn’t have done it without you!

 

[image src=”https://orientdb.com/wp-content/uploads/2015/12/5.png” align=”left” caption=”OrientDB Labs”] For the past few years all of us here at OrientDB have strived to deliver a fresh, innovative platform on which enterprises could work on to enrich their business.  We’ve always been proud of our work, and it’s been a real pleasure to collaborate with and learn from an ever growing community of dedicated developers.

With the launch of OrientDB back in 2011 we breathed some fresh air into the DBMS landscape, making waves with our multi-model approach (to the delight of some and shock of others). From then on it’s been a rollercoaster of a ride and in 2015 we continued to disrupt the market and climb the DBMS charts.  We’re hoping 2016 will be no different, and in that spirit we are launching our 2.2 Beta version of OrientDB in the next few days **queue the drum roll**

We couldn’t have achieved this without the ever growing community of developers who worked tirelessly alongside our team.  Of course, this is a Beta release so not suitable for production environments but we’re hoping our community is just as excited as we are with the announcement, and we’re looking forward to working together to test what we think will be our best release so far.[image src=”https://orientdb.com/wp-content/uploads/2016/02/portal1.png” align=”right”]

Our engineers have been working hard to make OrientDB even better and we’ve loaded version 2.2 full of new features which we’re sure you’ll love too.  The main focus of this version is on Security, APIs and performance, building on our distributed capabilities as well as graph enhancements.  We’ve added new features such as Teleporter, which introduces a simple, straightforward way to convert your relational databases to OrientDB.  Other new features include:

 

During the coming days, we’ll be discussing several of the new features here in our company blog so stay tuned for more on each of OrientDB’s new capabilities and enhancements.  Once we’re live, go ahead and take if for a test drive and let us know what you think.  For a complete list of features please refer to our OrientDB v2.2 documentation following the Beta launch.

Thank you to our community and looking forward to receiving your feedback!

 

Luca Garulli

Chief Executive Officer, Founder

February 8, 2016

By Andrey Lomakin, Lead Research & Development Engineer at OrientDB

In OrientDB v2.2 we’ve added tools which enable storage performance metrics to be gathered for a whole system and for a concrete command executed at that current moment. This feature will not only be interesting for database support teams, but it will probably also be of interest to users who want to understand why a database is fast or slow for their use case and what the reasoning is for results attained in a benchmark.

But before we consider characteristics gathered during storage profiling, let’s take a look at OrientDB’s architecture.

All high level OrientDB components, exposed to the user as clusters or indexes, are implemented inside the storage engine as “durable components” and extend the ODurableComponent class. This class is part of the framework created to make components/data structure operations atomic and isolated in terms of  ACID properties. Each durable component has to hold its data in direct memory, not in Java heap. But if in Java we operate variables to store/read application data, durable components operate pages.

A Page is a continuous snippet of memory which always has the same fixed size and is mapped to a file placed on a disk. When data is written to the page it is  automatically written to the file, but data is not written instantly, it must sometimes pass between the moment when data is written to the page and the moment data is written to the file.

We separate write operations on pages and the file system because file system operations are slow and we try to decouple data operations and file system operations. When we change the page it is not written to the disk instantly, as I have already mentioned above, but is placed in the write cache. The write cache aggregates all changed pages and stores them to the disk in a background thread in order of their file position. So, if we have changed pages with positions 3, 2, 8, 4, they will be stored in the order 2, 3, 4, 8.

Pages are sorted by their file positions because it does not matter whether you use DDR, SSD or HDD to store your data; sequential IO operations are always faster than random IO operations. Because pages are stored in a separate background thread, disk write operation speed will be decoupled from data modification operation speed.

In case of write operations we, may delay a data write and try to convert it to a sequential IO operation, but if we need to read data we need it instantly and can not delay the data read from file. So in this case we use the well known technique of caching frequently used pages in read cache.

So taking all of the above into account, you can see that OrientDB uses 2 caches:

When we read a page from a file, the following steps are performed:

When we modified the page content, it is automatically placed in the write cache.

There is one big problem with all those caches. Such system is not durable. If the application crashes, then some data which have not yet been written to the disk will be lost.

To avoid this kind of problems we use a database journal, aka WAL (write ahead log). This makes the whole process of writing of data a bit complex. When we modify the page we do not put the page in the write cache.  Instead, we write the difference of the page content into map keys which consist of a file and index of the changed page and values which contain diff of changes between original and changed page.

When an operation on a cluster or index is completed without exceptions we extract all changes from the map and log them inside the database journal and only after that do we apply those changes to the file pages and put them in the write cache. The database journal may be treated as an “append only” log so all write operations to the database journal are sequential and as result are fast. The process of writing changes to the database journal and applying them to the “real” file page is called “atomic operation commit”.

What value does the database journal give to us ?

In both cases data consistency will not be compromised.

Taking all of above into account you probably have already concluded that the main performance characteristics of OrientDB’s storage engine (and not only OrientDB)  are:

All those numbers will show us the direction our project must evolve towards. For example, if we have good numbers for a disk cache hit rate and very few pages are read for a single component operation, we have to improve disk cache speed as a whole.  However if we have a lot of page reads for a single component operation and very low numbers for page read speeds, we need to minimize the amount of pages accessed for the single operation and convert data structures to ones which uses more sequential rather than random IO operations.

Readers may ask: “well, all of this is very good but how it is related to us?”

The answer is: when you report performance issues please provide a benchmark (though we all have different hardware and sometimes cannot simply reproduce your issue) but also provide performance numbers gathered as the results of storage profiling.

Readers might also ask: “How is that done?”

It may be done in 2 ways, by using JMX and by using SQL commands.

The JMX console provides numbers gathered from the execution of all operations in storage but SQL commands provide data which are gathered for a selected set of commands.

To gather performance for a selected set of commands you can execute a script such as the one shown below:

At the end of the script you will see the following result:

As you can see you may see numbers for storage performance as a whole and numbers for the performance of each component.

Data from the atomic operation commit phase is presented as data from the component named “atomic operation”.

If you work with an embedded database you can start and stop storage profiling by calling the following methods:

OAbstractPaginatedStorage#startGatheringPerformanceStatisticForCurrentThread() to start storage profiling and OAbstractPaginatedStorage#completeGatheringPerformanceStatisticForCurrentThread() to stop profiling of storage.

You may also connect to the JMX server and read current performance numbers from MBean with name:

com.orientechnologies.orient.core.storage.impl.local.statistic:type=OStoragePerformanceStatisticMXBean,name=,id=

Hope it will be interesting for you to read overview of OrientDB architecture and performance characteristics which are important for us. Please do not forget to send results of profiling together with performance reports.

If you have any questions about this blog entry or about any of OrientDB’s features please post your question on stackoverflow and we will answer it.

Take your enterprise to the next level with OrientDB