污妖神尊
污妖神尊
2018-11-20  ·  2060 次阅读

Tinkerpop + JanusGraph + Hbase 环境搭建

192.168.1.2 master

安装配置好 jdk1.8 拥有一套正常运行的 Hadoop+Hbase+Zookeeper 集群。集群其他信息自定义。

注：文档中的主机 IP，主机名称，安装目录仅供参考，请以实际环境进行调整。

  • Apache TinkerPop™ 是图形数据库（OLTP）和图形分析系统（OLAP）的图形计算框架。
  • JanusGraph 是一个可扩展的图形数据库，专门用于存储和查询包含数千亿个分布在多机群集中的顶点和边缘的图形。
  • JanusGraph 是一个事务数据库，可以支持数千个并发用户实时执行复杂的图形遍历。

一、Hadoop+Hbase+Zookeeper 的集群搭建

此过程虽不麻烦，但一时讲清楚，还是很费时间的，在这就省略了。

二、创建目录，下载 Tinkerpop

根据 JanusGraph 官方文档 Table B.1. Version Compatibility Matrix 提供的信息, 在此次搭建中, 采用 Tinkerpop 3.2.7，JanusGraph 0.2.0。

  1. 创建目录
[root@master ~]# mkdir -p /usr/local/program
[root@master ~]# cd /usr/local/program/
[root@master program]# pwd
/usr/local/programe
  1. 下载 Tinkerpop
[root@master program]# wget http://www-eu.apache.org/dist/tinkerpop/3.2.7/apache-tinkerpop-gremlin-console-3.2.7-bin.zip # console, 相关地址http://tinkerpop.apache.org/.
[root@master program]# wget http://mirror.bit.edu.cn/apache/tinkerpop/3.2.7/apache-tinkerpop-gremlin-server-3.2.7-bin.zip # server,相关地址http://tinkerpop.apache.org/.
  1. 解压 Tinkerpop
[root@master program]# unzip apache-tinkerpop-gremlin-server-3.2.7-bin.zip
[root@master program]# unzip apache-tinkerpop-gremlin-console-3.2.7-bin.zip
[root@master program]# ls -1
apache-tinkerpop-gremlin-console-3.2.7
apache-tinkerpop-gremlin-server-3.2.7
apache-tinkerpop-gremlin-console-3.2.7-bin.zip
apache-tinkerpop-gremlin-server-3.2.7-bin.zip

三、在 Tinkerpop Server 上安装 JanusGraph 依赖

Note: 第四节有讲述如何配置 grapeConfig.xml 的内容，如果下载失败，可参考第四节。

  1. 进入 apache-tinkerpop-gremlin-server-3.2.7 下载依赖

安装方式： bin/gremlin-server.sh -i org.janusgraph janusgraph-all $VERSION，在这里下载 JanusGraph 的 0.2.0 版本。

[root@master apache-tinkerpop-gremlin-server-3.2.7]# bin/gremlin-server.sh -i org.janusgraph janusgraph-all 0.2.0
  1. 重要提示(Important, 这个地方应该重点显示, 为了好看, 就这样了)

下面引用 JanusGraph 官方文档中 7.4.2. Using TinkerPop Gremlin Server with JanusGraph 的一句话:

The above command uses Groovy Grape and if it is not configured properly download errors may ensue. Please refer to this section of the TinkerPop documentation for more information around setting up ~/.groovy/grapeConfig.xml.

相关地址 http://docs.janusgraph.org/latest/server.html 。

大意就是:

以上命令使用 Groovy Grape，如果没有正确配置，可能会出现下载错误。有关设置的更多信息，请参阅 TinkerPop 文档的这一部分~/.groovy/grapeConfig.xml。

上文提到的地址 http://tinkerpop.apache.org/docs/3.2.6/reference/#gremlin-applications 。

四、为避免下载依赖失败，修改 Groovy Grape

Note: Windows 目录: C:\Users\[User_Name]\.groovy; Linux 目录: ~/.groovy

修改内容如下：

  1. 在上述目录中创建 grapeConfig.xml 文件，或目录不存在，先创建目录。
  2. 修改文件内容如下：
<ivysettings>
 <settings defaultResolver="downloadGrapes"/>
 <property name="m2-pattern" value="${user.home}/.m2/repository/org/apache/tinkerpop/[module]/[revision]/[module]-[revision](-[classifier]).[ext]" />
 <property name="m2-pattern-ivy" value="${user.home}/.m2/repository/org/apache/tinkerpop/[module]/[revision]/[module]-[revision](-[classifier]).pom" />
 <caches>
 <cache name="nocache" useOrigin="true" />
 </caches>
 <resolvers>
 <chain name="downloadGrapes">
 <filesystem name="local-maven2" checkmodified="true" changingPattern=".*" changingMatcher="regexp" m2compatible="true" cache="nocache">
 <artifact pattern="${m2-pattern}"/>
 <ivy pattern="${m2-pattern-ivy}"/>
 </filesystem>
 <filesystem name="cachedGrapes">
 <ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/>
 <artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
 </filesystem>
 <ibiblio name="ibiblio" m2compatible="true"/>
 <ibiblio name="local" root="file:${user.home}/.m2/repository/" m2compatible="true"/>
 <ibiblio name="oracle" root="http://download.oracle.com/maven" m2compatible="true"/>
 </chain>
 </resolvers>
</ivysettings>
  1. 重新执行第三节的内容。

五、参数设置(master 节点操作)

  1. 在 ${TINKERPOP_HOME}/conf/ 目录创建 janusgraph-hbase-server.properties，添加内容如下：
gremlin.graph=org.janusgraph.core.JanusGraphFactory
storage.backend=hbase # 存储方式为Hbase.
storage.hostname=localhost # Zookeeper地址, 需要根据实际地址修改, 多台用','间隔.
# 以下信息可参考, 不影响下面步骤.
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.5 
# 以下信息可参考, 用于配置ElasticSearch索引信息, 不影响下面步骤.
index.search.backend=elasticsearch # 索引方式.
index.search.hostname=localhost # ElasticSearch主机地址, 多台用','间隔.
index.search.port=9200 # ElasticSearch通信端口.
index.search.elasticsearch.client-only=false
  1. 在 ${TINKERPOP_HOME}/conf/ 目录创建 janusgraph-gremlin-server.yaml，添加内容如下：
host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 300000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
graphs: {
 graph: conf/janusgraph-hbase-server.properties
}
plugins:
 - janusgraph.imports
scriptEngines: {
 gremlin-groovy: {
 imports: [java.lang.Math],
 staticImports: [java.lang.Math.PI],
 scripts: [scripts/empty-sample.groovy]}}
serializers:
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
 - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
 - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
 consoleReporter: {enabled: true, interval: 180000},
 csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
 jmxReporter: {enabled: true},
 slf4jReporter: {enabled: true, interval: 180000},
 gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
 graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536
  1. 在 ${TINKERPOP_HOME}/scripts/ 目录创建 empty-sample.groovy，(此文件默认存在，可直接使用。) 添加内容如下：
def globals = [:]
globals << [g : graph.traversal()]

六、运行 Tinkerpop Server

[root@master apache-tinkerpop-gremlin-server-3.2.7]# bin/gremlin-server.sh conf/janusgraph-gremlin-server.yaml
  1. 当显示如下内容时，说明启动成功：
......
INFO] GremlinServer - Executing start up LifeCycleHook
[INFO] Logger$info - Executed once at startup of Gremlin Server.
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
[WARN] AbstractChannelizer - The org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0 serialization class is deprecated.
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-lite with org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0
[INFO] AbstractChannelizer - Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0
[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 32 and boss thread pool of 1.
[INFO] GremlinServer$1 - Channel started at port 8182.
  1. 当提示以下内容时，修改文件，去除 \t
[ERROR] GremlinServer - Configuration file at conf/janusgraph-gremlin-server.yaml could not be found or parsed properly. [while scanning for the next token
found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB) for indentation)
 in 'reader', line 6, column 1:
 graph: conf/janusgraph-hbase-se ... 
 ^
]

七、通过 Tinkerpop Console 测试

  1. 进入 apache-tinkerpop-gremlin-console-3.2.7 目录，运行：
[root@master apache-tinkerpop-gremlin-console-3.2.7]# ./bin/gremlin.sh
Feb 08, 2018 4:57:21 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
 
 \,,,/
 (o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph
gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Configured localhost/127.0.0.1:8182
gremlin> :> g.V().count()
==> 0

运行无阻时，说明部署完成。可以继续往下进行。

举报
如果觉得这对你有用，请随意赞赏，给与作者支持
本站文章除注明转载外，均为本站原创或编译。转载请征得作者同意，并务必注明出处。
转载请注明：文章转载自 代码饭社区 [https://www.daimafans.com]
本文转载自：http://lshj.iteye.com/blog/2410594
评论 0
最新评论