Taiwan Hadoop Forum

台灣 Hadoop 技術討論區
現在的時間是 2018-04-22, 12:50

所有顯示的時間為 UTC + 8 小時




發表新文章 回覆主題  [ 2 篇文章 ] 
發表人 內容
 文章主題 : 提交應用程式到YARN上
文章發表於 : 2014-03-05, 15:44 
離線

註冊時間: 2013-07-29, 18:14
文章: 17
各位前輩大家好
我參考這網頁http://hortonworks.com/blog/how-to-deploy-memcached-on-yarn/
試著安裝memcached on YARN
遇到一些困難,我是用apache hadoop2.2.0且再安裝zookeeper 3.4.5
最後都是Application failed to complete success fully

container上的LOG內容為:
14/03/05 15:31:09 INFO yarn.ApplicationMaster: Initializing ApplicationMaster
14/03/05 15:31:09 INFO yarn.ApplicationMaster: Application master for app, appId=1, clustertimestamp=1394004574451, attemptId=1
14/03/05 15:31:09 INFO yarn.ApplicationMaster: Starting ApplicationMaster
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hadoop-2.2.0/workspace/tmp/nm-local-dir/usercache/hpds/appcache/application_1394004574451_0001/filecache/11/Runnable.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.2.0/workspace/tmp/nm-local-dir/usercache/hpds/appcache/application_1394004574451_0001/filecache/10/AppMaster.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:host.name=B17
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_21
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.class.path=$CLASSPATH:./Runnable.jar:./AppMaster.jar:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:./log4j.properties:./Runnable.jar:./AppMaster.jar:/opt/hadoop-2.2.0/etc/hadoop:/opt/hadoop-2.2.0/share/hadoop/common/hadoop-nfs-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0-tests.jar:/opt/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/xz-1.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/hadoop-annotations-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/guava-11.0.2.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-io-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jets3t-0.6.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/stax-api-1.0.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-math-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-el-1.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-logging-1.1.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/asm-3.2.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/avro-1.7.4.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/commons-lang-2.5.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/junit-4.8.2.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/activation-1.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/opt/hadoop-2.2.0/share/hadoop/common/lib/zookeeper-3.4.5.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0-tests.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-io-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-client-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-site-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-common-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-common-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-api-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/hadoop-annotations-2.2.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/commons-io-2.1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/hamcrest-core-1.1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/avro-1.7.4.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/junit-4.10.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:./log4j.properties
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.library.path=:/opt/hadoop-2.2.0/lib/native:/opt/hadoop-2.2.0/lib/native:/usr/java/packages/lib/amd64:/usr/lib/jni:/lib:/usr/lib
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:os.version=3.5.0-23-generic
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:user.name=hpds
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hpds
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/hadoop-2.2.0/workspace/tmp/nm-local-dir/usercache/hpds/appcache/application_1394004574451_0001/container_1394004574451_0001_01_000001
14/03/05 15:31:09 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=140.116.164.105:2181,140.116.164.106:2181 sessionTimeout=5000 watcher=zookeeper.groups.CreateGroup@6b7b9f29
14/03/05 15:31:10 INFO zookeeper.ClientCnxn: Opening socket connection to server hpds105.ee.ncku.edu.tw/140.116.164.105:2181. Will not attempt to authenticate using SASL (unknown error)
14/03/05 15:31:10 INFO zookeeper.ClientCnxn: Socket connection established to hpds105.ee.ncku.edu.tw/140.116.164.105:2181, initiating session
14/03/05 15:31:10 INFO zookeeper.ClientCnxn: Session establishment complete on server hpds105.ee.ncku.edu.tw/140.116.164.105:2181, sessionid = 0x1449111f01e0003, negotiated timeout = 5000
14/03/05 15:31:10 INFO zookeeper.ZooKeeper: Session: 0x1449111f01e0003 closed
14/03/05 15:31:10 INFO zookeeper.ClientCnxn: EventThread shut down
14/03/05 15:31:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/03/05 15:31:10 INFO client.RMProxy: Connecting to ResourceManager at /140.116.164.105:8081
14/03/05 15:31:10 INFO impl.NMClientAsyncImpl: Upper bound of the thread pool size is 500
14/03/05 15:31:10 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Max mem capabililty of resources in this cluster 40960
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Requested container ask: Capability[<memory:350, vCores:0>]Priority[0]
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Requested container ask: Capability[<memory:350, vCores:0>]Priority[0]
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Requested container ask: Capability[<memory:350, vCores:0>]Priority[0]
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Requested container ask: Capability[<memory:350, vCores:0>]Priority[0]
14/03/05 15:31:11 INFO yarn.ApplicationMaster: Requested container ask: Capability[<memory:350, vCores:0>]Priority[0]
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Got response from RM for container ask, allocatedCnt=2
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Launching shell command on a new container., containerId=container_1394004574451_0001_01_000002, containerNode=B18:41588, containerNodeURI=B18:8042, containerResourceMemory1024
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Launching shell command on a new container., containerId=container_1394004574451_0001_01_000003, containerNode=B17:42447, containerNodeURI=B17:8042, containerResourceMemory1024
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Setting up container launch container for containerid=container_1394004574451_0001_01_000002
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Setting up container launch container for containerid=container_1394004574451_0001_01_000003
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Use App Master jar from HDFS filesystem and add to local environment
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Use App Master jar from HDFS filesystem and add to local environment
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Set the environment for the application master
14/03/05 15:31:13 INFO yarn.ApplicationMaster: Set the environment for the application master
14/03/05 15:31:13 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1394004574451_0001_01_000003
14/03/05 15:31:13 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1394004574451_0001_01_000002
14/03/05 15:31:13 INFO impl.ContainerManagementProtocolProxy: Opening proxy : B17:42447
14/03/05 15:31:13 INFO impl.ContainerManagementProtocolProxy: Opening proxy : B18:41588
14/03/05 15:31:13 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_1394004574451_0001_01_000003
14/03/05 15:31:13 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_1394004574451_0001_01_000002
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Got response from RM for container ask, allocatedCnt=2
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Launching shell command on a new container., containerId=container_1394004574451_0001_01_000004, containerNode=B18:41588, containerNodeURI=B18:8042, containerResourceMemory1024
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Launching shell command on a new container., containerId=container_1394004574451_0001_01_000005, containerNode=B17:42447, containerNodeURI=B17:8042, containerResourceMemory1024
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Setting up container launch container for containerid=container_1394004574451_0001_01_000004
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Use App Master jar from HDFS filesystem and add to local environment
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Setting up container launch container for containerid=container_1394004574451_0001_01_000005
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Use App Master jar from HDFS filesystem and add to local environment
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Set the environment for the application master
14/03/05 15:31:14 INFO yarn.ApplicationMaster: Set the environment for the application master
14/03/05 15:31:14 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1394004574451_0001_01_000004
14/03/05 15:31:14 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1394004574451_0001_01_000005
14/03/05 15:31:14 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_1394004574451_0001_01_000005
14/03/05 15:31:14 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_1394004574451_0001_01_000004
14/03/05 15:31:15 INFO yarn.ApplicationMaster: Got response from RM for container ask, allocatedCnt=1
14/03/05 15:31:15 INFO yarn.ApplicationMaster: Launching shell command on a new container., containerId=container_1394004574451_0001_01_000006, containerNode=B18:41588, containerNodeURI=B18:8042, containerResourceMemory1024
14/03/05 15:31:15 INFO yarn.ApplicationMaster: Setting up container launch container for containerid=container_1394004574451_0001_01_000006
14/03/05 15:31:15 INFO yarn.ApplicationMaster: Use App Master jar from HDFS filesystem and add to local environment
14/03/05 15:31:15 INFO yarn.ApplicationMaster: Set the environment for the application master
14/03/05 15:31:15 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1394004574451_0001_01_000006
14/03/05 15:31:15 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_1394004574451_0001_01_000006
14/03/05 15:31:18 INFO yarn.ApplicationMaster: Got response from RM for container ask, completedCnt=1
14/03/05 15:31:18 INFO yarn.ApplicationMaster: Got container status for containerID=container_1394004574451_0001_01_000003, state=COMPLETE, exitStatus=143, diagnostics=Container [pid=901,containerID=container_1394004574451_0001_01_000003] is running beyond virtual memory limits. Current usage: 24.1 MB of 1 GB physical memory used; 3.0 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1394004574451_0001_01_000003 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 908 901 901 901 (java) 11 1 3135631360 5861 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 901 347 901 901 (bash) 0 0 102785024 304 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000003/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000003/stderr

Container killed on request. Exit code is 143

14/03/05 15:31:21 INFO yarn.ApplicationMaster: Got response from RM for container ask, completedCnt=4
14/03/05 15:31:21 INFO yarn.ApplicationMaster: Got container status for containerID=container_1394004574451_0001_01_000002, state=COMPLETE, exitStatus=143, diagnostics=Container [pid=29511,containerID=container_1394004574451_0001_01_000002] is running beyond virtual memory limits. Current usage: 24.1 MB of 1 GB physical memory used; 3.0 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1394004574451_0001_01_000002 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 29522 29511 29511 29511 (java) 11 1 3135606784 5860 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 29511 29251 29511 29511 (bash) 0 0 102789120 306 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000002/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000002/stderr

Container killed on request. Exit code is 143

14/03/05 15:31:21 INFO yarn.ApplicationMaster: Got container status for containerID=container_1394004574451_0001_01_000004, state=COMPLETE, exitStatus=143, diagnostics=Container [pid=29512,containerID=container_1394004574451_0001_01_000004] is running beyond virtual memory limits. Current usage: 24.2 MB of 1 GB physical memory used; 3.0 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1394004574451_0001_01_000004 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 29521 29512 29512 29512 (java) 11 1 3135606784 5894 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 29512 29251 29512 29512 (bash) 0 0 102789120 305 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000004/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000004/stderr

Container killed on request. Exit code is 143

14/03/05 15:31:21 INFO yarn.ApplicationMaster: Got container status for containerID=container_1394004574451_0001_01_000006, state=COMPLETE, exitStatus=143, diagnostics=Container [pid=29589,containerID=container_1394004574451_0001_01_000006] is running beyond virtual memory limits. Current usage: 24.2 MB of 1 GB physical memory used; 3.0 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1394004574451_0001_01_000006 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 29594 29589 29589 29589 (java) 10 1 3135606784 5900 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 29589 29251 29589 29589 (bash) 0 0 102789120 305 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000006/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000006/stderr

Container killed on request. Exit code is 143

14/03/05 15:31:21 INFO yarn.ApplicationMaster: Got container status for containerID=container_1394004574451_0001_01_000005, state=COMPLETE, exitStatus=143, diagnostics=Container [pid=975,containerID=container_1394004574451_0001_01_000005] is running beyond virtual memory limits. Current usage: 24.1 MB of 1 GB physical memory used; 3.0 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1394004574451_0001_01_000005 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 980 975 975 975 (java) 12 0 3135631360 5866 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 975 347 975 975 (bash) 0 0 102785024 303 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000005/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000005/stderr

Container killed on request. Exit code is 143

14/03/05 15:31:21 INFO yarn.ApplicationMaster: Application completed. Stopping running containers
14/03/05 15:31:21 INFO impl.ContainerManagementProtocolProxy: Closing proxy : B18:41588
14/03/05 15:31:21 INFO impl.ContainerManagementProtocolProxy: Closing proxy : B17:42447
14/03/05 15:31:21 INFO yarn.ApplicationMaster: Application completed. Signalling finish to RM
14/03/05 15:31:21 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=140.116.164.105:2181,140.116.164.106:2181 sessionTimeout=5000 watcher=zookeeper.groups.DeleteGroup@1e38b6fa
14/03/05 15:31:21 INFO zookeeper.ClientCnxn: Opening socket connection to server hpds106.ee.ncku.edu.tw/140.116.164.106:2181. Will not attempt to authenticate using SASL (unknown error)
14/03/05 15:31:21 INFO zookeeper.ClientCnxn: Socket connection established to hpds106.ee.ncku.edu.tw/140.116.164.106:2181, initiating session
14/03/05 15:31:21 INFO zookeeper.ClientCnxn: Session establishment complete on server hpds106.ee.ncku.edu.tw/140.116.164.106:2181, sessionid = 0x244911247d70002, negotiated timeout = 5000
14/03/05 15:31:21 INFO zookeeper.ZooKeeper: Session: 0x244911247d70002 closed
14/03/05 15:31:21 INFO zookeeper.ClientCnxn: EventThread shut down
14/03/05 15:31:21 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered.
14/03/05 15:31:21 INFO impl.AMRMClientAsyncImpl: Heartbeater interrupted
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:249)
14/03/05 15:31:21 INFO yarn.ApplicationMaster: Application Master failed. exiting
14/03/05 15:31:21 INFO impl.AMRMClientAsyncImpl: Interrupted while waiting for queue
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:275)


其中我有以
Dump of the process-tree for container_1394004574451_0001_01_000003 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 908 901 901 901 (java) 11 1 3135631360 5861 /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar
|- 901 347 901 901 (bash) 0 0 102785024 304 /bin/bash -c /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -jar Runnable.jar 1>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000003/stdout 2>/opt/hadoop-2.2.0/logs/userlogs/application_1394004574451_0001/container_1394004574451_0001_01_000003/stderr

Container killed on request. Exit code is 143

為關鍵字搜尋到http://stackoverflow.com/questions/20876498/unable-to-change-the-virtual-memory-of-hadoop-2-2-to-be-more-than-the-default-2
且將yarn.scheduler.maximum-allocation-mb
yarn.nodemanager.resource.memory-mb兩個值調高
但是還是一樣
不知道板上前輩們是否有遇過相同問題,可以替我解答
萬分感謝!


回頂端
 個人資料 E-mail  
 
 文章主題 : Re: 提交應用程式到YARN上
文章發表於 : 2014-03-05, 16:24 
離線

註冊時間: 2013-07-29, 18:14
文章: 17
目前已Container is running beyond memory limits為關鍵字
嘗試設定


回頂端
 個人資料 E-mail  
 
顯示文章 :  排序  
發表新文章 回覆主題  [ 2 篇文章 ] 

所有顯示的時間為 UTC + 8 小時


誰在線上

正在瀏覽這個版面的使用者:沒有註冊會員 和 1 位訪客


不能 在這個版面發表主題
不能 在這個版面回覆主題
不能 在這個版面編輯您的文章
不能 在這個版面刪除您的文章
不能 在這個版面上傳附加檔案

搜尋:
前往 :  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
正體中文語系由 竹貓星球 維護製作