{"id":225,"date":"2009-06-16T09:16:09","date_gmt":"2009-06-16T14:16:09","guid":{"rendered":"http:\/\/blog.espol.edu.ec\/hadoop\/?p=225"},"modified":"2009-06-16T09:16:09","modified_gmt":"2009-06-16T14:16:09","slug":"tip-de-rendimiento-reutilizar-la-jvm-entre-tareas-map","status":"publish","type":"post","link":"https:\/\/blog.espol.edu.ec\/hadoop\/2009\/06\/16\/tip-de-rendimiento-reutilizar-la-jvm-entre-tareas-map\/","title":{"rendered":"Tip de rendimiento: reutilizar la JVM entre tareas Map"},"content":{"rendered":"

En un e-mail de la lista core-user de hadoop, alguien pregunt\u00f3<\/a> lo siguiente:<\/p>\n

Subject:\u00a0Can I share datas for several map tasks?
\nHi,
\nI want to share some data structures for the map tasks on a same node(not through files), I mean, if one map task has already initialized some data structures (e.g. an array or a list), can other map tasks share these memorys and directly access them, for I don't want to reinitialize these datas and I want to save some memory. Can hadoop help me do this?<\/p><\/blockquote>\n

Eason.Lee sugiri\u00f3<\/a>:<\/p>\n

I think you can just define the data structures in your map classinit it in
\nsetup(Context context) and use it in your map method
\nhope it is helpful!<\/p><\/blockquote>\n

Pero si lo que se quiere es que los mappers que se levanten en el mismo nodo re-utilicen la estructura de datos creada por el primer Map task levantado en ese nodo, entonces la soluci\u00f3n\u2014planteada por Sharad Agarwal de Yahoo!<\/a>\u2014es re-utilizar la JVM:<\/p>\n

You can enable jvm reuse across tasks. See mapred.job.reuse.jvm.num.tasks in mapred-default.xml for usage. Then you can cache the data in a static variable in your mapper.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"

En un e-mail de la lista core-user de hadoop, alguien pregunt\u00f3 lo siguiente: Subject:\u00a0Can I share datas for several map tasks? Hi, I want to share some data structures for the map tasks on a same node(not through files), I mean, if one map task has already initialized some data structures (e.g. an array or […]<\/p>\n","protected":false},"author":1510,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[945,852],"tags":[2860,7885,5939],"class_list":["post-225","post","type-post","status-publish","format-standard","hentry","category-desarrollo","category-educacion","tag-hadoop","tag-jvm","tag-mapreduce"],"_links":{"self":[{"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/posts\/225","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/users\/1510"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/comments?post=225"}],"version-history":[{"count":2,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/posts\/225\/revisions"}],"predecessor-version":[{"id":227,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/posts\/225\/revisions\/227"}],"wp:attachment":[{"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/media?parent=225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/categories?post=225"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.espol.edu.ec\/hadoop\/wp-json\/wp\/v2\/tags?post=225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}