hadoop - Azkaban - HIve Job Failed -


i trying run hive job in azkaban

i able upload hive job hive job not getting executed properly.

error [hive-demo] [azkaban] failed build job executor job hive-demojob type     'hive' unrecognized. not construct job[{hive.query.01: drop table words;, hive.query.03: describe words;, hive.query.02: create table words (freq int, word string) row format delimited fields terminated ' ' stored textfile;, working.dir: /home/hduser/technology/azkaban/azkaban-executor/executions/44, azk.hive.action: execute.query, azkaban.job.attempt: 0, type: hive, hive.query.05: select * words limit   10;, hive.query.04: load data local inpath "res/input" table words;, user.to.proxy:   azkaban, hive.query.06: select freq, count(1) f2 words group freq sort f2 desc  limit 10;, parent = {azkaban.flow.flowid: hive-demo, azkaban.flow.execid: 44,  azkaban.flow.start.timezone: america/los_angeles, azkaban.flow.start.hour: 22,  azkaban.flow.start.second: 17, azkaban.flow.start.year: 2014,  azkaban.flow.start.milliseconds: 111, azkaban.flow.start.minute: 54,  azkaban.flow.start.timestamp: 2014-01-31t22:54:17.111-08:00, azkaban.flow.start.month: 01,  azkaban.flow.projectversion: 2, azkaban.flow.projectid: 4, azkaban.flow.uuid: 29a46293-cc95- 4c66-b2b1-51014205ce3b, azkaban.flow.start.day: 31, parent = {}}}] of type[hive]. 2014/02/01  12:24:17.295 +0530 error [hive-demo] [azkaban] failed build job type, skipping job    2014/02/01 12:24:17.295 +0530 error [hive-demo] [azkaban] job run failed! 2014/02/01 12:24:17.295 +0530 info [hive-demo] [azkaban] finishing job hive-demo @ 1391237657295  2014/02/01 12:24:17.382 +0530 info [flowrunnermanager] [azkaban] cleaning finished  2014/02/01 12:24:17.383 +0530 info [flowrunnermanager] [azkaban] cleaning execution 43  finished flows list. 2014/02/01 12:24:17.408 +0530 info [hive-demo] [azkaban] job  finished hive-demo status failed 2014/02/01 12:24:17.463 +0530 info [hive-demo]  [azkaban] finishing flow. awaiting termination 2014/02/01 12:24:17.463 +0530 info [hive- demo] [azkaban] setting flow status failed. 2014/02/01 12:24:17.464 +0530 info [hive- demo] [azkaban] flow set failed 2014/02/01 12:24:17.464 +0530 info [hive-demo]  [azkaban] setting end time flow 44 1391237657464 2014/02/01 12:24:17.552 +0530 info  [flowrunnermanager] [azkaban] flow 44 finished. adding finished flows  list. 2014/02/01 12:26:17.383 +0530 info [flowrunnermanager] [azkaban] cleaning  finished 2014/02/01 12:26:17.384 +0530 info [flowrunnermanager] [azkaban] cleaning execution  44 finished flows list.  ========================================================================

here content in plugin.properties

job.class=azkaban.jobtype.hiveutils.azkaban.hiveviaazkaban  hive.home=hive_home  hive.aux.jars.path=/usr/lib/hive/lib env.hive_home=$hive_home env.hive_aux_jars_path=/usr/lib/hive/lib  #hive.jvm.args=-dhive.querylog.location=. -dhive.exec.scratchdir=/tmp/hive-    ${user.to.proxy} -dhive.aux.jars.path=${hive.aux.jars.path} jobtype.global.jvm.args=${hive.jvm.args}

this private.properties jobtype.classpath=$hadoop_home/conf,$hadoop_home/lib/*,$hive_home/lib/*,$hive_home/conf,/usr /lib/hive/lib jobtype.class=azkaban.jobtype.hadoopjavajob

hive.aux.jars.path=/usr/lib/hive/lib env.hadoop_home=$hadoop_home env.hive_home=$hive_home env.hive_aux_jars_path=/usr/lib/hive/lib  hive.home=$hive_home  jobtype.global.classpath=${hive.classpath.items}   hive.classpath.items=./*,${hive.home}/lib/*,${hive.home}/conf/,${hive.aux.jars.path}/*,${had    oop.home}/*,${hadoop.home}/lib/*,${hadoop.home}/conf/

i have added plugin folder in executor folder only.

why not detecting job type. please help.

it's hard tell going on since neither share job nor logs, try following , let know if works. create file test.job following content:

type=hive azk.hive.action=execute.query hive.query=create table test(id int, name string) row format delimited fields terminated '\t'; user.to.proxy=hadoop

and submit described in 'creating flow' section. further details on different types of jobs azkaban (2.1) supports out of box see job types/plugin documentation.


-

Comments

Post a Comment

Popular posts from this blog

html - Sizing a high-res image (~8MB) to display entirely in a small div (circular, diameter 100px) -

first of all, thank being patient know simple question more couple of weeks' experience figure out. i've seen folks persevere , helpful on here , hope you're willing me. here fiddle: http://jsfiddle.net/65uzf/ here 1 of places searched answers on here: ( vertically center image thumbnails fix-sized div ) tried solutions on there , didn't work out me. i've searched ~4 or 5 others, , been unsuccessful those, too. what want take large image (what linked background-image #ysypic , #tabpic divs) , display thumbnail version of image in confines of divs have set it. is, using background properties have laid out in css, portion of image displays inside of circular divs. without these properties, display defaults smaller section of image [presumably pi*(50^2) area in top left] terrible resolution. i not care if entire image doesn't display inside of circles (it won't, given difference in shape), , overflow not being displayed fine. want able see figu
Read more

java - IntelliJ - No such instance method -

using intellij's 12 ultimate, i'm running following code in debugger: java import play.api.libs.json.jsvalue; public class foo { ... public jsvalue tojson() { ... } public class fooexample { ... foo foo = new foo(); system.out.println("...); //<-- breakpoint at breakpoint, right-clicked source code , picked "evaluate expression," typing in: foo.tojson() . but following error showed up: no such instance method: play.api.libs.json.jsvalue$class.com.foo.foo.tojson () am doing wrong? foo#tojson calls scala code, if matters. edit had breakpoint after instantiation of foo. downvoted it, deserved it. edit my answer no longer valid after op modified question (i.e. moved breakpoint actual is).. a breakpoint hit before line executed. in case foo has not yet been declared or instantiated (i.e. foo constructor has not yet been called). you'll need put breakpoint @ next line (or step on current line) if want evaluate foo .
Read more

identifier - Is it possible for an html5 document to have two ids? -

i wondering if html5 element can have 2 ids, this: <button id="thing1" id="thing2" name="button1" onclick="change()">click here</button> or this: <button id="thing1, thing2" name="button1" onclick="change()">click here</button> also, if 1 of ids said 1 thing, , other id said thing, 1 used? tl;dr : no. in versions of html, each element can have 1 id. see the spec : the id attribute specifies element's unique identifier (id). [dom] the value must unique amongst ids in element's home subtree , must contain @ least 1 character. the value must not contain space characters. note: there no other restrictions on form id can take; in particular, ids can consist of digits, start digit, start underscore, consist of punctuation , etc. note: element's unique identifier can used variety of purposes, notably way link specific parts of document usin
Read more