OrientDB Distributed Graph Database http://orientdb.com OrientDB is a Multi-Model NoSQL Database with a true Graph engine Thu, 28 Jul 2016 11:00:48 +0000 en-US hourly 1 Released OrientDB v2.2.6 http://orientdb.com/released-orientdb-2-2-6/ http://orientdb.com/released-orientdb-2-2-6/#comments Thu, 28 Jul 2016 08:28:07 +0000 http://orientdb.com/?p=15666 London, July 28, 2016 The OrientDB Team has just released OrientDB v2.2.6, resolving 11 issues from v2.2.5. If you are using 2.2.x series, please […]

The post Released OrientDB v2.2.6 appeared first on OrientDB Distributed Graph Database.

]]>
London, July 28, 2016

The OrientDB Team has just released OrientDB v2.2.6, resolving 11 issues from v2.2.5. If you are using 2.2.x series, please upgrade your production environments to v2.2.6. For more information, take a look at the Change Log.

Download OrientDB v2.2.6 now: http://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: http://orientdb.com/download-previous/

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

The post Released OrientDB v2.2.6 appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/released-orientdb-2-2-6/feed/ 0
Spatial Module in OrientDB 2.2 http://orientdb.com/spatial-module-orientdb-2-2/ http://orientdb.com/spatial-module-orientdb-2-2/#comments Mon, 25 Jul 2016 14:28:21 +0000 http://orientdb.com/?p=15637 London, July 25, 2016 In versions prior to 2.2, OrientDB had minimal support for storing and retrieving GeoSpatial data. The support was limited to […]

The post Spatial Module in OrientDB 2.2 appeared first on OrientDB Distributed Graph Database.

]]>
London, July 25, 2016

In versions prior to 2.2, OrientDB had minimal support for storing and retrieving GeoSpatial data. The support was limited to a pair of coordinates (latitude, longitude) stored as double in an OrientDB class, with the possibility to create a spatial index against those 2 coordinates in order to speed up a geo spatial query. So the support was limited to Point.
In OrientDB v.2.2 we created a brand new Spatial Module with support for different types of Geometry objects stored as embedded objects in a user defined class

  • Point (OPoint)
  • Line (OLine)
  • Polygon (OPolygon)
  • MultiPoint (OMultiPoint)
  • MultiLine (OMultiline)
  • MultiPolygon (OMultiPlygon)
  • Geometry Collections

Along with those data types, the module extends OrientDB SQL with a subset of SQL-MM functions in order to support spatial data.The module only supports EPSG:4326 as Spatial Reference System. This blog post is an introduction to the OrientDB spatial Module, with some examples of its new capabilities. You can find the installation guide here.

Let’s start by loading some data into OrientDB. The dataset is about points of interest in Italy taken from here. Since the format is ShapeFile we used QGis to export the dataset in CSV format (geometry format in WKT) and import the CSV into OrientDB with the ETL in the class Points and the type geometry field is OPoint.

image08

Since the WKT field is in string format we have to create the geometry property in the Points class

create property Points.location EMBEDDED OPoint

 
And update it in order to insert the geometry field by using the Function ST_GeomFromText.

update Points set location = ST_GeomFromText(WKT)

 
In this way, we store the position in the field location stored as an Embedded object with an OPoint class and field coordinates. If we want to insert another point of interest we can use an SQL statement in the following way:

INSERT INTO Points SET name = “Some Name”, location = {"@class": "OPoint","coordinates" : [lon,lat]}

 
The order of lon/lat here in order to be GeoJSON compliant.

Alternetively, we may use the function to create the Point object from the WKT format:

INSERT INTO Points SET name = “Some Name”, location = St_GeomFromText("POINT (lon lat)")

 
Now let’s suppose that we are in the middle of Rome near Santa Maria in Trastevere, thirsty, and looking for something to drink. We could use the function ST_Distance_Sphere to find points that are near us within a max distance of, let’s say, 200 meters.

select *,ST_Distance_Sphere(location,ST_GeomFromText('POINT(12.4696635 41.8894657)')) as distance from Points where ST_Distance_Sphere(location,ST_GeomFromText('POINT(12.4696635 41.8894657)')) < 200 order by distance

 
image07

We can see that at ~113 meters distance we have a drinking water fountain. By using the location field in the result set, we can easily use the Google Maps API to create a map from that data.

image04

Now we want to know the same information but faster than before. Just create a spatial index on the location property:

create index Points.location on Points(location) SPATIAL ENGINE LUCENE

 
And execute the query again.

image05

Let’s complicate things a little bit. We said the new spatial module is able to handle different type of Geometry objects. With the same procedure as before (ShapeFile + QGis + CSV + ETL) we import the dataset of states and provinces boundaries of countries into OrientDB.
In this case the target class is ‘Provinces’ and the type geometry field is OMultipolygon, created with SQL

create property Provinces.geometry EMBEDDED OMultipolygon

 
Then we inspect some provinces of Italy just to check the imported data.

Once we have the provinces of Italy, we can use a spatial function to execute a nice query and extract interesting information.

For example we could use the ST_Contains function if we want know which is the Province of Calcata, a tiny little town near Rome.

select iso_3166_2,gn_name,region,geonunit from Provinces where ST_Contains(geometry, ST_GeomFromText('POINT(12.42617 42.21952)')) = true

 
image06

Or we could use the ST_Intersects function to get all the provinces that the Tiber river crosses. For this query, we extracted the WKT geometry data of the Tiber (MultiLineString) from here.

select iso_3166_2,gn_name,region,geonunit from Provinces where ST_Intersects(geometry, ST_GeomFromText(<MultiLineString>) = true order by region

 
where <MultiLineString> is the WKT notation of the Tiber

At this point, it is also possible to create a map with the provincial boundaries.

There are other implemented functions bundled with the module. You can find the full documentation here. The Spatial Module is a new feature in OrientDB v.2.2, if you want to see it improved with new functionalities, please drop us a line using OrientDB’s issue tracker.

Hoping this comes in handy,

Enrico Risa
OrientDB LTD

The post Spatial Module in OrientDB 2.2 appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/spatial-module-orientdb-2-2/feed/ 0
Released OrientDB v2.2.5 http://orientdb.com/released-orientdb-2-2-5/ http://orientdb.com/released-orientdb-2-2-5/#comments Wed, 20 Jul 2016 15:47:44 +0000 http://orientdb.com/?p=15617 London, July 20, 2016 The OrientDB Team has just released OrientDB v2.2.5, resolving 12 issues from v2.2.4. If you are using 2.2.x series, please upgrade […]

The post Released OrientDB v2.2.5 appeared first on OrientDB Distributed Graph Database.

]]>
London, July 20, 2016

The OrientDB Team has just released OrientDB v2.2.5, resolving 12 issues from v2.2.4. If you are using 2.2.x series, please upgrade your production environments to v2.2.5. For more information, take a look at the Change Log.

Download OrientDB v2.2.5 now: http://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: http://orientdb.com/download-previous/

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

The post Released OrientDB v2.2.5 appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/released-orientdb-2-2-5/feed/ 0
Released OrientDB 2.2.4 http://orientdb.com/released-orientdb-2-2-4/ http://orientdb.com/released-orientdb-2-2-4/#comments Sat, 09 Jul 2016 09:30:00 +0000 http://orientdb.com/?p=15444 London, July 8, 2016 The OrientDB Team has just released OrientDB v2.2.4, resolving 26 issues from v2.2.3. If you are using 2.2.x series, please upgrade […]

The post Released OrientDB 2.2.4 appeared first on OrientDB Distributed Graph Database.

]]>
London, July 8, 2016

The OrientDB Team has just released OrientDB v2.2.4, resolving 26 issues from v2.2.3. If you are using 2.2.x series, please upgrade your production environments to v2.2.4. For more information, take a look at the Change Log.

Download OrientDB v2.2.4 now: http://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: http://orientdb.com/download-previous/

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

The post Released OrientDB 2.2.4 appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/released-orientdb-2-2-4/feed/ 0
Using the Spark Connector for OrientDB http://orientdb.com/spark-orientdb/ http://orientdb.com/spark-orientdb/#comments Fri, 08 Jul 2016 12:50:42 +0000 http://orientdb.com/?p=15383 London, July 8, 2016 By Andrea Iacono The Spark connector for OrientDB has been provided by Metreta and hosted on github at https://github.com/metreta/spark-orientdb-connector, letting Spark […]

The post Using the Spark Connector for OrientDB appeared first on OrientDB Distributed Graph Database.

]]>
London, July 8, 2016
By Andrea Iacono

The Spark connector for OrientDB has been provided by Metreta and hosted on github at https://github.com/metreta/spark-orientdb-connector, letting Spark and OrientDB interoperate in two ways: accessing OrientDB data from Spark and writing Spark data to OrientDB. The connector is also aware of the difference between an OrientDB document database and an OrientDB graph database:

  • Using OrientDB as a documents database: The OrientDB documents can be read from/written to Spark RDDs.
  • Using OrientDB as a graph database: The vertices and edges can be read from/written to Spark GraphX graphs.

To compile the connector, clone the master branch and update its build.sbt file with the Scala version and the Spark version you’re using. You may subsequently launch the package command on sbt:

sbt package

 

Upon performing these steps, you should find a jar file containing the compiled connector in your target directory. Be sure to have created the test database as well (as shown in the connector’s page).

The first step for creating our sample project is to create a build.sbt, where we have to define the library dependencies:

libraryDependencies ++= Seq(
 "com.orientechnologies" % "orientdb-core" % "2.2.3",
 "com.orientechnologies" % "orientdb-client" % "2.2.3",
 "com.orientechnologies" % "orientdb-graphdb" % "2.2.3",
 "com.orientechnologies" % "orientdb-distributed" % "2.2.3",
 "org.apache.spark" % "spark-core_2.11" % "1.6.1",
 "org.apache.spark" % "spark-graphx_2.11" % "1.6.1",
 "org.scala-lang" % "scala-compiler" % "2.11.4",
 "org.scala-lang" % "scala-library" % "2.11.4",
 "org.scala-lang" % "scala-reflect" % "2.11.4",
 "jline" % "jline" % "2.12",
 "com.tinkerpop.blueprints" % "blueprints-core" % "2.6.0",
 "com.fasterxml.jackson.core" % "jackson-databind" % "2.7.4",
 "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.7.4"
)

 

We must then configure Spark to attach to OrientDB, which we can do by defining the SparkConf in the following way:

  val conf = new SparkConf()
    .setMaster("local[*]")
    .setAppName("ConnectorSample")
    .set("spark.orientdb.clustermode", "remote")
    .set("spark.orientdb.connection.nodes", "127.0.0.1")
    .set("spark.orientdb.protocol", "remote")
    .set("spark.orientdb.dbname", "test")
    .set("spark.orientdb.port", "2424")
    .set("spark.orientdb.user", "admin")
    .set("spark.orientdb.password", "admin")

 

We can now share data between Spark and OrientDB.

Orient Documents to/from Spark RDDs
Let’s start reading some OrientDB documents as a Spark RDD:

var peopleRdd: RDD[OrientDocument] = sc.orientQuery("Person")

 

With the orientQuery() method, we can read the documents of a class from OrientDB and may have them as a Spark RDD, on which we can do the usual manipulations. We can then save them back to OrientDB:

peopleRdd
 .filter(person => person.getString("name") == "John")
 .map(person => new Person("Foo", "Bar"))
 .saveToOrient("Person")

 

Like in this example where, after a bit of manipulation, we use the saveToOrient() method to save all the elements of the RDD as OrientDB documents, we can check both querying OrientDB via Studio or querying from the code:

sc.orientQuery("Person").foreach(p => println(s"Person: ${p.getString("surname")}, ${p.getString("name")}"))

 

We can also update the OrientDB documents using the upsertToOrient() method, as shown in this example where we update a document’s property via the RDD and save them back to OrientDB:

peopleRdd
 .filter(person => !person.getString("surname").startsWith("New"))
 .map(person => new Person(person.getString("name"), "New " + person.getString("surname")))
 .upsertToOrient("Person")

 

Orient Graphs to/from Spark GraphX
When we deal with graphs, RDDs are not enough and so we must move to Spark’s API for graph computing: GraphX.

To access OrientDB vertices and edges, we must use the orientGraph() method as shown in this example:

val peopleGraph: Graph[OrientDocument, OrientDocument] = sc.orientGraph()

 

Since peopleGraph is a org.apache.spark.graphx.Graph object, we can use its methods to access OrientDB data, as in these examples:

val people: VertexRDD[OrientDocument] = peopleGraph.vertices
val relationships: EdgeRDD[OrientDocument] = peopleGraph.edges

println(s"The graph contains ${people.count()} vertices and ${relationships.count()} edges.\n")

 

We can also access triplets, as in this example where we print friendships among people:

peopleGraph
 .triplets
 .foreach(triplet => {
   val srcPerson: OrientDocument = triplet.srcAttr
   val dstPerson: OrientDocument = triplet.dstAttr
   println(s"Person: ${srcPerson.getString("surname")}, ${srcPerson.getString("name")} [${triplet.srcId}]. Friend: ${dstPerson.getString("surname")}, ${dstPerson.getString("name")} [${triplet.dstId}]")
 })

 

The built-in graph algorithms supplied by GraphX are also available, like the triangleCount() used here to show the triangles among people:

val triangles = peopleGraph.triangleCount()

// prints how many triangles every vertex participate in
triangles
 .vertices
 .foreach {
   case (vertexId, trianglesNumber) => println(s"Person [${vertexId}] participates in ${trianglesNumber} triangles.")
 }

 

When we have a GraphX graph and we want to save it as an OrientDB graph, we can use the saveGraphToOrient():

val gr: Graph[Person, String] = createSampleGraph(sc)
gr.saveGraphToOrient()

 

In this example, the createSampleGraph() method just creates a simple graph with three vertices and five edges as RDDs and then builds the graph upon them:

def createSampleGraph(sparkContext: SparkContext): Graph[Person, String] = {

 val people: RDD[(VertexId, Person)] =
   sparkContext.parallelize(
     Array(
       (1L, new Person("Alice", "Anderson")),
       (2L, new Person("Bob", "Brown")),
       (3L, new Person("Carol", "Clark"))
     )
   )


 val edges: RDD[Edge[String]] =
   sparkContext.parallelize(
     Array(
       Edge(1L, 2L, "Friendship"),
       Edge(1L, 3L, "Friendship"),
       Edge(2L, 1L, "Friendship"),
       Edge(3L, 1L, "Friendship"),
       Edge(3L, 2L, "Friendship")
     )
   )
 Graph(people, edges)
}

 

This full code of these examples is available on github at https://github.com/andreaiacono/SparkOrientDbConnectorDemo.

The post Using the Spark Connector for OrientDB appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/spark-orientdb/feed/ 0
Configuring your OrientDB Teleporter Migration (Part 2) http://orientdb.com/teleporter-configuration-part2/ http://orientdb.com/teleporter-configuration-part2/#comments Wed, 06 Jul 2016 13:20:43 +0000 http://orientdb.com/?p=15352 London, July 6, 2016 In the previous blog post we analyzed the reasons that may lead us to configure a migration performed through OrientDB […]

The post Configuring your OrientDB Teleporter Migration (Part 2) appeared first on OrientDB Distributed Graph Database.

]]>
London, July 6, 2016

In the previous blog post we analyzed the reasons that may lead us to configure a migration performed through OrientDB Teleporter.  We have mainly seen that the issue with missing foreign keys’ definition can be overcome thanks to the mapping between logical Relationships of the source database and the Edge classes of the Graph Model. Today we’ll see that we can exploit Teleporter’s configuration even further.

Thanks to this configuration you can:

  • Add new Edge classes in the Graph Model even though Relationships are missing.
  • Modify existing mappings between Relationships and Edge classes. In fact, if the Relationship you are mapping is already present in your database, it will be overridden with the parameters you defined in the configuration.In this way you can change the name, the direction and the properties of the Edge class correspondent to a Relationship already present in the database schema.
  • Define Edge classes for join tables during a migration exploiting the aggregation strategy.

Last time we analyzed a use case where we exploited the first feature using mapping, in order to avoid that missing foreign keys in the database schema could lead to a graph database without edges.
By analyzing another use case in this blog post, we will demonstrate how take advantage of the other two functionalities.

In fact, in some instances we may want to change the default names chosen by Teleporter for the Edge classes, as they are inferred starting from the attributes’ names and they may not be so significant; we might need to add or remove properties to a specific Edge class, or want to modify the attributes of an “aggregator-edge” representing complex Relationships N-N, direction included.

Let’s suppose we have the following E-R Model:

image00

 

Then let’s presume all the foreign keys are defined in the database schema at the physical layer.
image02

 

Let’s suppose we want adopt the aggregation strategy for the migration towards OrientDB in order to aggregate the Project_Employee join table. In this way, the join table is not converted into a Vertex class but into an appropriate Edge class, and each field not involved in any relationship with other tables is aggregated in the properties of the new built edge.

Even though the new Graph Model does not reflect the original E-R Model, the aggregation leads to a great saving in terms of resources and avoids a substantial overhead as the aggregation process simplifies the structure of the resulting Graph Model, producing a simpler graph. In fact, each record in the join table will not correspond to a vertex, but an edge between two vertices.

If you want to know more about the aggregation strategy, you can refer to the documentation or you can read this article.

Therefore, performing a direct aggregating migration without any configuration we will obtain the following graph Model:

image01

 

Now suppose we want to map the following Relationships and Edge classes in a different way:

  • Aggregate-Relationship between Project and Employee: we want to invert the direction of the Project2Employee Edge class and consequently change its name to WorksAtProject. Moreover, we want to add a new role property to the Edge class properties list.
  • Relationship between Employee and Department: we want to have WorksIn as the new name for the HasDepartment Edge class and add a new since property to the Edge class.

Regarding the Relationships between Department and City, we want to maintain the default mapping, so we will not configure anything about them.

Consequently, it’s important to allow for the configuration to override the default mapping inferred and executed by Teleporter: so if you must simply change a few details in the final graph model, you will not have to write down the complete configuration, but you may just specify what you want to change.

This is the full configuration we must submit to Teleporter in order to achieve the set goals:

{
  "edges": [{
    "WorksIn": {
      "mapping": {
        "fromTable": "EMPLOYEE",
        "fromColumns": ["department"],
        "toTable": "DEPARTMENT",
        "toColumns": ["id"],
        "direction": "direct"
      },
      "properties": {
        "since": {
          "type": "DATE",
          "mandatory": true,
          "readOnly": false,
          "notNull": false
        }
      }
    },
    "WorksAtProject": {
      "mapping": {
        "fromTable": "PROJECT",
        "fromColumns": ["id"],
        "toTable": "EMPLOYEE",
        "toColumns": ["id"],
        "joinTable": {
          "tableName": "PROJECT_EMPLOYEE",
          "fromColumns": ["project_id"],
          "toColumns": ["employee_id"]
        },
        "direction": "inverse"
      },
      "properties": {
        "role": {
          "type": "STRING",
          "mandatory": true,
          "readOnly": false,
          "notNull": true
        }
      }
    }
  }]
}

 

Let’s take a closer look at each Relationship-Edge class mapping.

1. Relationship: Employee-Department, cardinality 1-N.

 "WorksIn": {
      "mapping": {
        "fromTable": "EMPLOYEE",
        "fromColumns": ["department"],
        "toTable": "DEPARTMENT",
        "toColumns": ["id"],
        "direction": "direct"
      },
      "properties": {
        "since": {
          "type": "DATE",
          "mandatory": true,
          "readOnly": false,
          "notNull": false
        }
      }
    }

 

In the configuration above we have defined which parameters must to be overridden. We decided to change the name of the Edge class through the WorksIn field, add a since property with a DATE type and maintain the original direction from Employee vertices to Department vertices.

Teleporter will recognize the Relationship you want to override on the basis of the values:

  • fromTable
  • fromColumns
  • toTable
  • toColumns

These values must to be coherent with the direction of the foreign key representing the above-mentioned Relationship. Otherwise Teleporter will interpret the relationship as a new one. So, if we want override the Relationship starting from Employee and ending with the Department Vertex class through the following configuration:

    "WorksIn": {
      "mapping": {
        "fromTable": "DEPARTMENT",
        "fromColumns": ["id"],
        "toTable": "EMPLOYEE",
        "toColumns": ["department"],
        "direction": "direct"
      }, … 

 

As result we would obtain the adding of a second relationship with an inverted direction between the two tables. Therefore, our graph model would have two Edge classes where the second one is totally wrong.
So remember to be coherent with the underlying physical schema during the mapping definition.

If you want to reverse the Edge, do not use the fromTable and toTable fields, instead take advantage of the direction field using “inverse” as value.

2. Relationship: Project-Employee, cardinality N-N.

 "WorksAtProject": {
      "mapping": {
        "fromTable": "PROJECT",
        "fromColumns": ["id"],
        "toTable": "EMPLOYEE",
        "toColumns": ["id"],
        "joinTable": {
          "tableName": "PROJECT_EMPLOYEE",
          "fromColumns": ["project_id"],
          "toColumns": ["employee_id"]
        },
        "direction": "inverse"
      },
      "properties": {
        "role": {
          "type": "STRING",
          "mandatory": true,
          "readOnly": false,
          "notNull": true
        }
      }
    }

 

The aggregation during the migration is performed on join tables which only allow join operations between two tables.

Each candidate join table is converted into an appropriate Edge class, and each attribute of the table itself is aggregated into the properties of the new built Edge. We can define this “aggregator-edge” both if foreign keys between joint table and external tables are defined or not.

If no foreign keys are defined you can kill two birds with one stone. In fact, you can declare the two Relationships with the external tables and define the mapping with an Aggregator-Edge class in one shot. Otherwise, if foreign keys are present in the schema you can act on the mapping anyway. In our example, we changed the direction of the Edge class and consequently, through the WorksAtProject field, its name as well.

The implicit direction of the Edge is set according to the fromTable and toTable fields, setting them as follows:

  • “fromTable”: “PROJECT”
  • “toTable”: “EMPLOYEE”

We decided to express the relationship between Projects and Employees through edges starting from Project vertices and ending into Employee vertices.
But we are even free to exploit the semantics of the direction field in order to reverse the direction of the final Edge, conveniently renamed WorksAtProject.
Moreover we added a role property of type STRING using the syntax already adopted for the previous mapping.

As shown in the example, in order to activate the aggregation you must define the additional joinTable field:

 "joinTable": {
          "tableName": "PROJECT-EMPLOYEE",
          "fromColumns": ["project_id"],
          "toColumns": ["employee_id"]
  }

 

This info is essential for Teleporter to infer all the single relationships between the records belonging to the “EMPLOYEE” and “PROJECT” tables and to coherently build all the edges:

  • tableName: the name of the join table which will be aggregated into the declared Edge.
  • fromColumns: the join table’s columns involved in the relationship with the “fromTable”.
  • toColumns: the join table’s columns involved in the relationship with the “toTable”.

Submitting this configuration to Teleporter, we will finally obtain the following Graph Model:

image03

 

That’s it, we achieved our goals by acting directly on the mapping between the Relationships of the source database and the Edge classes of the OrientDB graph database.

Now you are ready to start with your own data!

Stay tuned,

Gabriele Ponzi
OrientDB Ltd

References

Teleporter documentation index:
https://github.com/orientechnologies/orientdb-docs/blob/master/Teleporter-Index.md

Teleporter configuration:
https://github.com/orientechnologies/orientdb-docs/blob/master/Teleporter-Import-Configuration.md

The post Configuring your OrientDB Teleporter Migration (Part 2) appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/teleporter-configuration-part2/feed/ 0
Using OrientDB with Scala – By Andrea Iacono http://orientdb.com/using-orientdb-scala/ http://orientdb.com/using-orientdb-scala/#comments Wed, 29 Jun 2016 15:08:21 +0000 http://orientdb.com/?p=15228 By OrientDB Software Engineer Andrea Iacono There’s no specific Scala API for OrientDB but we can use the Java API, paying attention to the […]

The post Using OrientDB with Scala – By Andrea Iacono appeared first on OrientDB Distributed Graph Database.

]]>
By OrientDB Software Engineer Andrea Iacono

There’s no specific Scala API for OrientDB but we can use the Java API, paying attention to the different calling conventions between Java and Scala.

Let’s start defining our SBT project. To use OrientDB we just need one library dependency of OrientDB, so the build.sbt file will be like this:

name := "OrientDbScalaExample"
version := "1.0"
scalaVersion := "2.11.8"

libraryDependencies ++= Seq(
"com.orientechnologies" % "orientdb-graphdb" % "2.2.3",
)

 

In this example we’ll only use the Graph API so we only need that dependency in our build.sbt, though a list of all the available OrientDB libraries is present in the Supported Libraries documentation page.

Once we have our build.sbt we can start using OrientDB. Let’s start creating a new database, using the Graph API:

val uri: String = "plocal:target/database/scala_sample"
val factory: OrientGraphFactory = new OrientGraphFactory(uri)
val graph: OrientGraph = factory.getTx()

 

If the database is not existing it will be created and opened, while if it already exists, it will simply be opened. Refer to the GraphFactory documentation

Now that we have access to our graph database, we can define some new classes, extending V and E. Let’s start defining a Person class, extending V that has a first and a last name:

val person: OrientVertexType = graph.createVertexType("Person")
person.createProperty("firstName", OType.STRING)
person.createProperty("lastName", OType.STRING)

 

Then we define other two classes, Company and Project:

val company: OrientVertexType = graph.createVertexType("Company")
company.createProperty("name", OType.STRING)
company.createProperty("revenue", OType.LONG)

val project: OrientVertexType = graph.createVertexType("Project")
project.createProperty("name", OType.STRING)

 

We also want to extend the E class, to create a work relationship between a Person and a Company:

val work: OrientEdgeType = graph.createEdgeType("Work")
work.createProperty("startDate", OType.DATE)
work.createProperty("endDate", OType.DATE)
work.createProperty("projects", OType.LINKSET)

 

This class has three properties:

  • A startDate, which tells us when the person started to work for that company.
  • An endDate, which tells us when the person stopped working for that company. If this property is not present, we assume the person is still working for the company.
  • A LinkSet of projects, which are all the project the person contributed to while working for the company. See Types documentation for more details about the LinkSet.

Now that we’ve defined the classes we need, we can add some vertices; let’s start with the Person John Doe:

val johnDoe: Vertex = graph.addVertex("class:Person", Nil: _*)
johnDoe.setProperty("firstName", "John")
johnDoe.setProperty("lastName", "Doe")

 

The second parameter passed to the addVertex() method is due to the difference in calling conventions between Java and Scala; that parameter is needed to let the Scala compiler understand which of the overloaded versions of the method to invoke.

Setting properties can also be done by specifying the property name and value as pairs when calling the addVertex() method itself, as seen in the following examples where we create two other people, a company and two projects:

val johnSmith: Vertex = graph.addVertex("class:Person", "firstName", "John", "lastName", "Smith")
val janeDoe: Vertex = graph.addVertex("class:Person", "firstName", "Jane", "lastName", "Doe")
val acme: Vertex = graph.addVertex("class:Company", "name", "ACME", "revenue", "10000000")
val acmeGlue: Vertex = graph.addVertex("class:Project", "name", "ACME Glue")
val acmeRocket: Vertex = graph.addVertex("class:Project", "name", "ACME Rocket")

 

Now we’ll create a couple of edges to connect our vertices; we can do it using the vertex itself:

val johnSmithAcme: Edge = johnSmith.addEdge("Work", acme) 
johnSmithAcme.setProperty("startDate", "2009-01-01")

 

Or we can use the instance of the OrientGraph and the method addEdge():

val johnDoeAcme: Edge = graph.addEdge(null, johnDoe, acme, "Work")
johnDoeAcme.setProperty("startDate", "2010-01-01")
johnDoeAcme.setProperty("endDate", "2013-04-21")
johnDoeAcme.setProperty("projects", Set(acmeGlue, acmeRocket))

 

In this case, beside the start and end of the work properties, we’ve also set the projects that the person contributed to while working for that company.
For more details about managing vertices and edges, have a look at the Vertices and Edges documentation.

Now we have a graph database like this:

OrientDB graph database-scala

Note that in the image there are no edges among the projects and the people because we defined the relationship “person contributed to a project” not as an edge but as a LinkSet in the “work relationship” edge.

Now that our database contains some data we can use OrientDB’s extended SQL to query that data. Let’s suppose we want to find all the people who work for ACME. The query will be something like this:

SELECT expand(in('Work')) FROM Company WHERE name='ACME'

 

So, we can launch the query with the command() and then the execute() method:

val res: OrientDynaElementIterable = graph
          .command(new OCommandSQL(s"SELECT expand(in('Work')) FROM Company WHERE name='ACME'"))
          .execute()

 

which returns an Iterable that we can use to loop over the results and to print some info:

res.foreach(v => {

            // casts the element in the Iterable to an OrientVertex
            val person = v.asInstanceOf[OrientVertex]

            // gets the "Work" out edge of the Vertex
            val workEdgeIterator = person.getEdges(Direction.OUT, "Work").iterator()
            val edge = workEdgeIterator.next()

            // retrieves info about person analyzing the "Work" edge
            val status = if (edge.getProperty("endDate") != null) "retired" else "active"
            val projects =
                if (edge.getProperty("projects") != null)
                  edge
                    .getProperty("projects")
                    .asInstanceOf[Set[Vertex]]
                    .map(v=>v.getProperty[String]("name"))
                    .mkString(", ")
                else
                    "Any project"

            // and prints them
            println(s"Name: ${person.getProperty("lastName")}, " +
                 s"${person.getProperty("firstName")} " +
                 s"[${status}]. Worked on: ${projects}.")
})

 

In the loop we cast the element of the Iterable to an OrientVertex (which is OrientDB’s implementation of the interface Vertex); we then call the getEdges() method on the vertex (with the Direction.OUT and “Work” params) to retrieve the outgoing edges with label “Work” of that vertex; since we populated our Person vertices with only one edge, we get that edge directly calling the next() method on the Iterator.
We use the getProperty() method to retrieve the value of a specific property; in the case of the “projects” property, which is of type LinkSet, we have to cast it to a Set of Vertices to create a list of the names of the projects using map.

Running this sample will produce this output:

ACME people:
Name: Doe, John [retired]. Worked on: ACME Glue, ACME Rocket.
Name: Smith, John [active]. Worked on: Any project.

 

This is a very simple example on how to start using Scala: if you want to fully exploit the potential of OrientDB, refer to the Official Java API documentation.

Here is the full code:

import com.orientechnologies.orient.core.metadata.schema.OType
import com.orientechnologies.orient.core.sql.OCommandSQL
import com.tinkerpop.blueprints.{Direction, Edge, Vertex}
import com.tinkerpop.blueprints.impls.orient._

import scala.collection.JavaConversions._

object OrientDbSample extends App {

   val WorkEdgeLabel = "Work"

   // opens the DB (if not existing, it will create it)
   val uri: String = "plocal:target/database/scala_sample"
   val factory: OrientGraphFactory = new OrientGraphFactory(uri)
   val graph: OrientGraph = factory.getTx()

   try {

       // if the database does not contain the classes we need (it was just created),
       // then adds them
       if (graph.getVertexType("Person") == null) {

           // we now extend the Vertex class for Person and Company
           val person: OrientVertexType = graph.createVertexType("Person")
           person.createProperty("firstName", OType.STRING)
           person.createProperty("lastName", OType.STRING)

           val company: OrientVertexType = graph.createVertexType("Company")
           company.createProperty("name", OType.STRING)
           company.createProperty("revenue", OType.LONG)

           val project: OrientVertexType = graph.createVertexType("Project")
           project.createProperty("name", OType.STRING)

           // we now extend the Edge class for a "Work" relationship
           // between Person and Company
           val work: OrientEdgeType = graph.createEdgeType(WorkEdgeLabel)
           work.createProperty("startDate", OType.DATE)
           work.createProperty("endDate", OType.DATE)
           work.createProperty("projects", OType.LINKSET)
       }
       else {

           // cleans up the DB since it was already created in a preceding run
           graph.command(new OCommandSQL("DELETE VERTEX V")).execute()
           graph.command(new OCommandSQL("DELETE EDGE E")).execute()
       }

       // adds some people
       // (we have to force a vararg call in addVertex() method to avoid ambiguous
       // reference compile error, which is pretty ugly)
       val johnDoe: Vertex = graph.addVertex("class:Person", Nil: _*)
       johnDoe.setProperty("firstName", "John")
       johnDoe.setProperty("lastName", "Doe")

       // we can also set properties directly in the constructor call
       val johnSmith: Vertex = graph.addVertex("class:Person", "firstName", "John", "lastName", "Smith")
       val janeDoe:Vertex=graph.addVertex("class:Person","firstName","Jane","lastName","Doe")

       // creates a Company
       val acme : Vertex = graph.addVertex("class:Company","name","ACME","revenue","10000000")

       // creates a couple of projects
       val acmeGlue: Vertex = graph.addVertex("class:Project", "name", "ACME Glue")
       val acmeRocket: Vertex = graph.addVertex("class:Project", "name", "ACME Rocket")

       // creates edge JohnDoe worked for ACME
       val johnDoeAcme: Edge = graph.addEdge(null, johnDoe, acme, WorkEdgeLabel)
       johnDoeAcme.setProperty("startDate", "2010-01-01")
       johnDoeAcme.setProperty("endDate", "2013-04-21")
       johnDoeAcme.setProperty("projects", Set(acmeGlue, acmeRocket))

       // another way to create an edge, starting from the source vertex
       val johnSmithAcme: Edge = johnSmith.addEdge(WorkEdgeLabel, acme)
       johnSmithAcme.setProperty("startDate", "2009-01-01")

       // prints all the people who works/worked for ACME
       val res: OrientDynaElementIterable = graph
         .command(new OCommandSQL(s"SELECT expand(in('${WorkEdgeLabel}')) FROM Company WHERE name='ACME'"))
         .execute()

       println("ACME people:")
       res.foreach(v => {

           // gets the person
           val person : OrientVertex = v.asInstanceOf[OrientVertex]

           // gets the "Work" edge
           val workEdgeIterator = person.getEdges(Direction.OUT, WorkEdgeLabel).iterator()
           val edge = workEdgeIterator.next()

           // retrieves worker's info
           val status = if (edge.getProperty("endDate") != null) "retired" else "active"
           val projects =
             if (edge.getProperty("projects") != null)
               edge
                 .getProperty("projects")
                 .asInstanceOf[Set[Vertex]]
                 .map(v=>v.getProperty[String]("name"))
                 .mkString(", ")
             else
               "Any project"

           // and prints them
           println(s"Name: ${person.getProperty("lastName")}, " +
                         s"${person.getProperty("firstName")} " +
                         s"[${status}]. Worked on: ${projects}.")
       })
   }
   finally {
       graph.shutdown()
   }
}

The post Using OrientDB with Scala – By Andrea Iacono appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/using-orientdb-scala/feed/ 0
OrientDB Announces .NET Partnership with Innov8tive http://orientdb.com/innov8tive-orientdb/ http://orientdb.com/innov8tive-orientdb/#comments Mon, 27 Jun 2016 14:21:46 +0000 http://orientdb.com/?p=15125 London, June 27, 2016   &     We are proud to announce that Innov8tive and OrientDB, have joined forces to develop the new .NET drivers for […]

The post OrientDB Announces .NET Partnership with Innov8tive appeared first on OrientDB Distributed Graph Database.

]]>
London, June 27, 2016

ISCLogo-Horizontal  &  orientdb_logocrop

 

We are proud to announce that Innov8tive and OrientDB, have joined forces to develop the new .NET drivers for OrientDB.  This exciting venture means that OrientDB users will soon be provided with official .NET drivers and support, which was under active community development.

“Innov8tive is passionate about everything from mobile apps to data centers.  Furthermore, their strong focus on security makes them an ideal partner for OrientDB.”

- Luca Garulli, CEO, OrientDB

Innov8tive, experts in the field of high end software development and consulting services as well as certified Microsoft partners, have helped turn complex software ideas into reality and power IT departments with their experienced team of consultants.

“OrientDB has always been an integral part of our tool set. We are delighted to be able to deepen our partnership with OrientDB and enhance the capabilities of the .NET driver.”

- Gray Delacluyse, CPDO, Innov8tive

“Innov8tive is passionate about everything from mobile apps to data centers.  Furthermore, their strong focus on security makes them an ideal partner for OrientDB.” Says Luca Garulli, CEO for OrientDB.  “This is exactly the type of company OrientDB seeks for their ‘Think Globally, Act Locally’ Partnership program.”

Driver development is currently in progress and will be officially announced by OrientDB.  Users will soon be able to have improved cross-platform development, which will only enhance applications built using OrientDB.  This is especially true for OrientDB 2.2.x users who enjoy improved security features as well as an optimized core engine along with new configurable graph consistency.

If you’re interested in using OrientDB but haven’t downloaded it already, give it a try.  OrientDB community edition is Open Source and completely free.  Its Enterprise Edition may be used at no cost for development purposes and competitive support packages come with a commercial license for those hoping to use OrientDB Enterprise edition in production environments.

Paolo Puccini
Marketing Specialist
OrientDB LTD

The post OrientDB Announces .NET Partnership with Innov8tive appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/innov8tive-orientdb/feed/ 0
Configuring your OrientDB Teleporter Migration (Part 1) http://orientdb.com/teleporter-configuration/ http://orientdb.com/teleporter-configuration/#comments Wed, 22 Jun 2016 14:09:36 +0000 http://orientdb.com/?p=15056 London, June 22, 2016 The latest version of OrientDB Teleporter makes it possible to define the configuration necessary in order to manage and customize […]

The post Configuring your OrientDB Teleporter Migration (Part 1) appeared first on OrientDB Distributed Graph Database.

]]>
London, June 22, 2016

The latest version of OrientDB Teleporter makes it possible to define the configuration necessary in order to manage and customize your migration. Configuration is performed in order to affect the mapping between the schema of your source database and the schema of the target graph database in OrientDB. The first feature implementation enables one to intervene on the relationships of your source relational database and the resultant edges in the graph database.

Why is the configuration that maps the correspondence between relationships and edges so important? To answer this question it’s important to know how Teleporter works, in particular how it infers relationships present in the source database and how it builds a correspondent edge for each of them in the OrientDB graph schema. The first 2 steps performed by Teleporter are the mapping between the E-R Model of the source database and the building of a coherent Graph Model. If you want to know more about these steps you may refer to the first two blog posts or the official documentation.

 

E-R and Graph Model

 

All the info required to build the E-R model is fetched from the physical definition of the source database schema, then each element is translated semi-directly into the Graph Model as follows:

  • Each Entity is converted into a Vertex class.
  • Each Relationship between two Entities is converted into an Edge class.

Entities are inferred from tables defined in your schema, Relationships from foreign keys’ definitions. Therefore if you didn’t defined any foreign key between the tables on which you usually perform join operations, Relationships will not be recognized nor built and, as result, no Edge classes will be present in your final Graph Database. This can result in an incorrect or ineffective migration. For this reason, it’s essential to exploit this feature and provide proper configuration for Teleporter’s execution, thus ensuring that constraints missing in your database metadata are overcome.

Thanks to the configuration you can:

  • Add new Edge classes in the Graph Model even though relationships are missing.
  • Modify existing mappings between Relationships and Edge classes.
  • Define Edge classes for join tables during a migration exploiting the aggregation strategy.

In this article we will see how the first goal is achieved through the study of a simple user case. The other two scenarios will be analyzed in subsequent blog posts.

Let’s suppose we have the following E-R Model:

 

E-R Model

 

As you can see below, no foreign keys are defined at the physical layer between the Employee and Department tables, nor are there any defined between Department and City tables.

 

No Key Definition in Tables

 

With this approach, Relationships are lost during the Graph Model building, obtaining only the Vertex classes correspondent to the tables, but no Edge classes will be defined:

 

OrientDB Teleporter: no edge definition

 

So we must define relationships not declared in the database metadata and map them to the correspondent edges through the configuration. Let’s start creating a configuration file in JSON format.

The root field is edges, which contains an array of elements. Here we must define all the edges we want to have in our final graph. In fact, each element in the array is an Edge class definition, containing the mapping with a Relationship in the relational model.

 

This is the full configuration which must be submitted to Teleporter in order to achieve our goal.

{
  "edges": [{
   "LocatedIn": {
    "mapping": {
     "fromTable": "DEPARTMENT",
     "fromColumns": ["location"],
     "toTable": "CITY",
     "toColumns": ["id"],
     "direction": "direct"
   },
    "properties": {
     "since": {
      "type": "DATE",
      "mandatory": true,
      "readOnly": false,
      "notNull": false
    }
   }
  }
 }, {
   "WorksIn": {
    "mapping": {
     "fromTable": "EMPLOYEE",
     "fromColumns": ["department"],
     "toTable": "DEPARTMENT",
     "toColumns": ["id"],
     "direction": "direct"
  }
 }
}, {
  "IsManagerFor": {
   "mapping": {
    "fromTable": "DEPARTMENT",
    "fromColumns": ["manager"],
    "toTable": "EMPLOYEE",
    "toColumns": ["id"],
    "direction": "inverse"
   }
  }
 }]
}

 

Let’s take a closer look at each Relationship-Edge class mapping.

1. Relationship: Department-City, cardinality 1-N.

 

{
 "LocatedIn": {
  "mapping": {
   "fromTable": "DEPARTMENT",
   "fromColumns": ["location"],
   "toTable": "CITY",
   "toColumns": ["id"],
   "direction": "direct"
  },
  "properties": {
   "since": {
    "type": "DATE",
    "mandatory": true,
    "readOnly": false,
    "notNull": false
   }
  }
 }
}

 

The first field, LocatedIn, will be used as the name for the Edge class in the OrientDB graph. Inside it, we have two fields: mapping and properties. The mapping field enables the mapping of the Relationship with the LocatedIn Edge class on the basis of 4 essential values:

  • fromTable: The foreign entity that imports the primary key of the parent table. See the “DEPARTMENT” table in our example.
  • fromColumns: The attributes involved in the foreign key. See the “location” column in the “DEPARTMENT” table from our example.
  • toTable: The parent entity whose primary key is imported by the foreign table. See the “CITY” table in our example.
  • toColumns: The attributes involved in the primary key imported. See the “id” column in the “CITY” table from our example.

As a direction we chose direct, because we want edges to have the same direction of the Relationship, starting from Department and ending in City.

Moreover, we can define additional properties for the final Edge class, LocatedIn, by using the properties field.

 

"properties": {
    "since": {
    "type": "DATE",
    "mandatory": true,
    "readOnly": false,
    "notNull": false
  }

 

In this case, we have to set the name of the property (“since” in our example), and its attributes:

  • type: The OrientDB type. This value is mandatory. If not declared, the property is not added to the Edge.
  • mandatory: Adds the mandatory constraint to the property and applies to it the specified value (true or false).
  • readOnly: Adds the readOnly constraint to the property and applies to it the specified value (true or false).
  • notNull: Adds the notNull constraint to the property and applies to it the specified value (true or false).

So, in the example above we added a property named “since” of type OType.DATE to our Edge class.

2. Relationship: Employee-Department, cardinality 1-N.

 

{
  "WorksIn": {
   "mapping": {
    "fromTable": "EMPLOYEE",
    "fromColumns": ["department"],
    "toTable": "DEPARTMENT",
    "toColumns": ["id"],
    "direction": "direct"
  }
 }
}

 

Here we defined an Edge class for the Relationship between Employee and Department, choosing WorksIn as name. We did not declare any property for this Edge class.

3. Relationship: Department-Employee, cardinality 1-1.

 

{
 "IsManagerFor": {
  "mapping": {
   "fromTable": "DEPARTMENT",
   "fromColumns": ["manager"],
   "toTable": "EMPLOYEE",
   "toColumns": ["id"],
   "direction": "inverse"
  }
 }
}

 

In this case we are mapping the opposite Relationship between Department and Employee: this relationship specifies who is the manager for a specific department.
Suppose we want the Edge class to represent the opposite direction with respect to the Relationship. In order to achieve this, we use the inverse value in the direction field as well as coherent semantics for the Edge class name: thus, the edges belonging to the IsManagerFor class will start from the Employee vertices and will end into Department vertices.

Thanks to this configuration, we obtain the following Graph Model:

 

configured graph model with OrientDB Teleporter

 

That’s all, now we must submit the configuration file to Teleporter indicating the path of the JSON file through the argument -conf:

 

./oteleporter.sh -jdriver postgresql -jurl jdbc:postgresql://localhost:5432/testdb
          -juser username -jpasswd password
          -ourl plocal:$ORIENTDB_HOME/databases/testdb
          -conf /tmp/migration-config.json

 

After the first migration, the graph database will be built and the configuration you passed as argument to Teleporter will be copied into the database folder in a path such as the one below:

 

$ORIENDB_HOME/databases/testdb/teleporter-config/migration-config.json 

 

In the following executions, the new configuration in your database will be processed automatically, making it coherent and simplifying the synchronization procedure. If you want to change any settings you can modify the file directly.
In fact Teleporter, at execution time, sequentially looks for:

  1. the configuration file migration-config.json in the database directory $ORIENDB_HOME/databases/testdb/teleporter-config/
  2. if no config file is found, then a potential input config will be considered
  3. if no config file is passed as argument, the migration will be performed without any configuration

In the next blog post we will see how to modify existing relationships and use the configuration to our advantage with the aggregation strategy, so stay tuned!

References

Teleporter documentation index:
https://github.com/orientechnologies/orientdb-docs/blob/master/Teleporter-Index.md

Teleporter configuration:
https://github.com/orientechnologies/orientdb-docs/blob/master/Teleporter-Import-Configuration.md

Gabriele Ponzi
OrientDB LTD

The post Configuring your OrientDB Teleporter Migration (Part 1) appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/teleporter-configuration/feed/ 0
Released OrientDB 2.2.3 http://orientdb.com/released-orientdb-2-2-3/ http://orientdb.com/released-orientdb-2-2-3/#comments Mon, 20 Jun 2016 14:59:42 +0000 http://orientdb.com/?p=15050 London, June 20, 2016 The OrientDB Team has just released OrientDB v2.2.3, resolving 7 issues from v2.2.2. If you are still using 2.2.x (or 2.1 […]

The post Released OrientDB 2.2.3 appeared first on OrientDB Distributed Graph Database.

]]>
London, June 20, 2016

The OrientDB Team has just released OrientDB v2.2.3, resolving 7 issues from v2.2.2. If you are still using 2.2.x (or 2.1 series), please upgrade your production environments to v2.2.3. For more information, take a look at the Change Log.

Download OrientDB v2.2.3 now: http://orientdb.com/download

If you are currently using a previous version of OrientDB, we recommend you upgrade using the link above. However, if you would like to download previous versions of OrientDB Community Edition, you may do so here: http://orientdb.com/download-previous/

A big thank you goes out to the OrientDB team and all the contributors who worked hard on this release, providing pull requests, tests, issues and comments.

Best regards,

Luigi Dell’Aquila
Director of Consulting
OrientDB LTD

The post Released OrientDB 2.2.3 appeared first on OrientDB Distributed Graph Database.

]]>
http://orientdb.com/released-orientdb-2-2-3/feed/ 0