{"id":638,"date":"2011-03-18T22:38:04","date_gmt":"2011-03-18T14:38:04","guid":{"rendered":"http:\/\/www.strongd.net\/?p=638"},"modified":"2011-03-18T22:38:04","modified_gmt":"2011-03-18T14:38:04","slug":"%e8%ae%a9%e4%bb%a3%e7%a0%81%e5%8f%96%e4%bb%a3%e4%bd%a0%e7%9a%84%e9%85%8d%e7%bd%ae%e6%96%87%e4%bb%b6%e5%90%a7","status":"publish","type":"post","link":"https:\/\/www.strongd.net\/?p=638","title":{"rendered":"\u8ba9\u4ee3\u7801\u53d6\u4ee3\u4f60\u7684\u914d\u7f6e\u6587\u4ef6\u5427"},"content":{"rendered":"

\u6700\u8fd1, \u5728\u7f16\u5199\u4e00\u4e2a\u4e13\u95e8\u538b\u6d4bNameNode\u7684\u5de5\u5177(\u4ee5\u4e0b\u7b80\u79f0s4nn), \u5b83\u6709\u4e24\u4e2a\u96be\u70b9 :<\/p>\n

    \n
  1. s4nn\u9700\u8981\u53ef\u4ee5\u6a21\u62df\u4e0a\u4e07\u4e2aDataNode ;<\/li>\n
  2. s4nn \u9700\u8981\u7075\u6d3b\u7684\u652f\u6301\u5bf9NameNode\u8bbf\u95ee\u884c\u4e3a\u7684\u5b9a\u4e49.<\/li>\n<\/ol>\n

    \u540e\u8005\u5bfc\u81f4\u4e86\u672c\u6587\u7684\u601d\u8003.<\/p>\n

    \u547d\u4ee4\u884c\u53c2\u6570\u548c\u914d\u7f6e\u6587\u4ef6\u662f\u6700\u5e38\u7528\u6765\u914d\u7f6e\u7cfb\u7edf\u7684\u65b9\u6cd5, \u524d\u8005\u7528\u4e8e\u914d\u7f6e\u9879\u8f83\u5c11, \u540e\u8005\u5219\u9002\u5408\u914d\u7f6e\u590d\u6742\u60c5\u51b5. \u8fd9\u4e24\u79cd\u65b9\u5f0f\u90fd\u6709\u5171\u540c\u4ee4\u4eba\u75db\u82e6\u7684\u5730\u65b9:<\/p>\n

      \n
    1. \u7f16\u5199\u4ee3\u7801\u53bb\u8f7d\u5165->\u89e3\u6790->\u8f6c\u6362, \u901a\u5e38\u5982\u540c\u5904\u7406\u534f\u8bae\u822c\u65e0\u804a(\u8981\u662f\u6709\u4e2a\u4ec0\u4e48\u53d8\u66f4, KMN!!);<\/li>\n
    2. \u5bf9\u4e8e\u590d\u6742\u7684\u914d\u7f6e\u6587\u4ef6\u7f16\u5199\u800c\u8a00, \u603b\u662f\u6ca1\u6709\u987a\u624b\u7684\u7f16\u8f91\u5668\u652f\u6301, \u5199\u8d77\u6765\u65e2\u7d2f\u53c8\u6613\u9519.<\/li>\n<\/ol>\n

      \u8981\u662f\u7528\u4ee3\u7801\u53d6\u4ee3\u914d\u7f6e\u6587\u4ef6\u5462?<\/p>\n

      \u5443\u2026 \u8fd9\u4f1a\u5f88\u9ebb\u70e6\u5427, \u50cfjava\u8fd9\u6837\u7684\u6539\u4e86\u4ee3\u7801\u90a3\u8fd8\u4e0d\u8981\u91cd\u65b0\u7f16\u8bd1\u554a?<\/p>\n

      \u55ef, \u786e\u5b9e, \u4f46\u5e76\u6ca1\u6709\u60f3\u8c61\u4e2d\u90a3\u4e48\u9ebb\u70e6, \u4e00\u4e9b\u6280\u5de7\u53ef\u4ee5\u8ba9\u5b83\u53d8\u5f97\u7b80\u5355(\u81f3\u5c11java\u662f\u8fd9\u6837), \u5982Btrace<\/a>.<\/p>\n

      \u7528\u4ee3\u7801\u53d6\u4ee3\u72ec\u7acb\u7684\u914d\u7f6e\u6587\u4ef6\u4e0d\u662f\u65b0\u9c9c\u7684\u505a\u6cd5, \u50cfGuice<\/a>,\u00a0Gant<\/a> \u7b49\u5df2\u7ecf\u4ee5Internal DSL\u7684\u4ee3\u7801\u4ee3\u66ff\u4e86XML. \u597d\u5904\u5f88\u660e\u663e:<\/p>\n

        \n
      1. \u826f\u597d\u7684DSL\u98ce\u683c, \u7b80\u6d01\u6613\u61c2 ;<\/li>\n
      2. \u514d\u53bb\u5bf9\u914d\u7f6e\u6587\u4ef6\u7684\u89e3\u6790\u8f6c\u6362 ;<\/li>\n
      3. \u6700\u597d\u7684\u7f16\u8f91\u5668\u652f\u6301, \u8bed\u6cd5\u9ad8\u4eae, \u4e00\u952e\u683c\u5f0f\u5316, \u63d0\u793a\u8865\u5168, \u91cd\u6784 ;<\/li>\n
      4. \u7f16\u8bd1\u5668\u5e2e\u52a9\u67e5\u9519;<\/li>\n
      5. \u4e0e\u4ee3\u7801\u65e0\u7f1d\u7ed3\u5408, \u80fd\u591f\u5bb9\u6613\u5728\u53d8\u5316\u4e2d\u4fdd\u6301\u4e00\u81f4\u6027.<\/li>\n<\/ol>\n

        \u7b80\u8a00\u4e4b\u4e24\u4e2a\u5b57, \u201c\u9ad8\u6548\u201d! \u8fd9\u5012\u662f\u633a\u9002\u5408s4nn\u7684, \u4e0d\u59a8\u4e00\u8bd5!<\/p>\n

        \u4ece\u9700\u6c42\u7684\u89d2\u5ea6\u51fa\u53d1, \u914d\u7f6e\u5e94\u8be5\u80fd\u591f\u5b8c\u6210:<\/p>\n

          \n
        1. \u5b9a\u4e49\u4e00\u7ec4Client RPC\u8c03\u7528\u884c\u4e3a, \u5176\u8c03\u7528\u53c2\u6570, \u6b21\u6570;<\/li>\n
        2. \u5b9a\u4e49DataNode\u7684\u8fd0\u884c\u65f6\u7279\u5f81, \u4e2a\u6570.<\/li>\n<\/ol>\n

          \u4e3a\u6ee1\u8db3\u8fd9\u4e24\u70b9, \u4ee3\u7801\u8bbe\u8ba1\u4e0a\u53ef\u80fd\u4f1a\u6709:<\/p>\n

          class ClientDriver {\r\n  void addRpc(times, name, args)\r\n  void setNameNode(address)\r\n}\r\n\r\nclass DataNodeSimulator {\r\n  void setCapacity(size)\r\n  void setHeartbeatInterval(sec)\r\n  void setBlockReportInterval(sec)\r\n  void setNameNode(address)\r\n}<\/pre>\n
          \u90a3\u4e48\u5176\u914d\u7f6e\u6587\u4ef6\u4f1a\u662f:<\/p>\n
           1 <configuration>\r\n 2   <client>\r\n 3     <rpc times=\"100000\" name=\"getFileInfo\">\r\n 4       <arguments>\r\n 5         <param name=\"src\">\/foo<\/param>\r\n 6       <\/arguments>\r\n 7     <\/rpc>\r\n 8   <\/client>\r\n 9   <namenode>\r\n10     <address>host:port<\/address>\r\n11   <\/namenode>\r\n12   <datanode>\r\n13     <simulator num=\"10000\">\r\n14       <capacity unit=\"GB\">200<\/capacity>\r\n15       <heartbeatInterval unit=\"SECOND\">1<\/heartbeatInterval>\r\n16       <blockReportInterval unit=\"HOUR\">1<\/blockReportInterval>\r\n17     <\/simulator>\r\n18   <\/datanode>\r\n19 <\/configuration><\/pre>\n
          \u7b80\u5355\u5417? \u7b80\u5355, \u90a3\u662f\u56e0\u4e3a\u8fd9\u53ea\u662f\u6700\u57fa\u672c\u7684, \u5b9e\u9645\u7684\u914d\u7f6e\u5e94\u8003\u8651\u5230:<\/p>\n
            \n
          1. \u670930\u79cdRPC\u65b9\u6cd5, \u5176\u4e2d\u53c2\u6570\u4e2a\u6570\u6700\u591a\u7684\u67096, \u00a0\u53c2\u6570\u7c7b\u578b\u5e76\u975e\u90fd\u662f\u57fa\u672c\u7c7b\u578b, \u53c2\u6570\u503c\u53ef\u80fd\u9700\u8981\u6309\u7167\u67d0\u79cd\u89c4\u5219\u968f\u5373\u751f\u6210;<\/li>\n
          2. DataNode\u6a21\u62df\u5668\u4e5f\u4f1a\u6709\u53ef\u80fd\u9700\u8981\u652f\u6301\u591a\u79cd\u9009\u9879\u7ec4\u5408, \u5982\u5b9e\u9645\u96c6\u7fa4\u79cd\u4e0d\u662f\u6240\u6709\u7684\u673a\u5668\u7684\u5bb9\u91cf\u90fd\u4e00\u6837\u7684\u7b49.<\/li>\n<\/ol>\n
            \u8fd9\u6837\u7684XML\u4f1a\u81c3\u80bf\u5230\u4ec0\u4e48\u7a0b\u5ea6\u2026<\/p>\n
            \u597d\u5427, \u770b\u770b\u7528\u4ee3\u7801\u7684\u6548\u679c:<\/p>\n
             1 import com.taobao.s4nn.*;\r\n 2\r\n 3 public class Main extends Bootstrap {\r\n 5\r\n 6   protected void config() {\r\n 7     setConcurrency(16);\r\n 8     setMaxRandomPathDepth(10);\r\n 9\r\n10     \/*\r\n11      * DataNode simulator config\r\n12      *\/\r\n13     simulate(3, new DataNodeSimulatorGenerator() {\r\n14       protected void config() {\r\n15         setCapacity(gibibyte(1));\r\n16         setTickPeriodSecond(1);\r\n17         setHeartbeatInterval(1);\r\n18         setBlockReportInterval(3);\r\n19         setNameNode(proxyOfNameNodeAt(\"localhost:10001\"));\r\n20       }\r\n21     });\r\n22\r\n23     \/*\r\n24      * NameNode Status pre-setting config\r\n25      *\/\r\n26     parallel(1, new StatusPresetterBuilder() {\r\n27       protected void config() {\r\n28         setName(client(\"InitalStatus\"));\r\n29         setNamenode(proxyOfNameNodeAt(\"localhost:10001\"));\r\n30         times(1, create(randomSrc, defaultFsPermission, overwrite(true), replication((short) 3), blocks(1)));\r\n31       }\r\n32     });\r\n33\r\n34     \/*\r\n35      * Client RPC driver config\r\n36      *\/\r\n37     parallel(10, new ClientDriverBuilder() {\r\n38       protected void config() {\r\n39         setName(seqNameWith(\"client\"));\r\n40         setNamenode(proxyOfNameNodeAt(\"localhost:10001\"));\r\n41\r\n42         times(1, getBlockLocations(src(\"\/foo\"), randomOffsetIn(0, 1024), randomLengthIn(0, 2048)));\r\n43         times(2, getListing(src(\"\/foo\")));\r\n44         times(3, getLocatedListing(src(\"\/foo\")));\r\n45         times(4, getStats());\r\n46         times(5, getDatanodeReport(all));\r\n47         times(6, getPreferriedBlockSize(src(\"\/foo\")));\r\n48         times(7, getFileInfo(src(\"\/foo\")));\r\n49         times(8, getContentSummary(src(\"\/foo\")));\r\n50         times(9, setOwner(src(\"\/foo\"), username(\"jushi\"), groupname(\"dwbasis\")));\r\n51         times(10, setReplication(src(\"\/foo\"), replication((short) 4)));\r\n52         times(11, setPermission(src(\"\/foo\"), defaultFsPermission));\r\n53         times(12, rename(src(\"\/foo\"), src(\"\/bar\")));\r\n54         times(13, mkdirs(src(\"\/bar\"), defaultFsPermission));\r\n55         times(14, setQuota(src(\"\/foo\"), randomNamespaceQuotaIn(1024, 2048), randomDiskspaceQuotaIn(1024, 4096)));\r\n56         times(15, setTime(src(\"\/foo\"), currentTimeMillis, unchanged));\r\n57       }\r\n58     });\r\n59   }\r\n60 }<\/pre>\n
            \u55ef, \u00a0\u65e2\u4fdd\u6301\u6613\u8bfb\u6027\u53c8\u66f4\u4e3a\u7b80\u6d01, \u00a0\u91cd\u70b9\u662f\u4e0d\u7528\u518d\u5199\u90a3\u4e9b\u989d\u5916\u5904\u7406XML\u7684\u4ee3\u7801\u4e86.<\/p>\n
            \u8fd9\u91cc\u6548\u4eff\u4e86Guice\u4e2dAbstractModule\u7684DSL\u505a\u6cd5, \u63d0\u4f9b\u4e86setXxx, times\u7b49\u5185\u7f6e\u65b9\u6cd5, \u7528Java IDE\u7f16\u5199\u8d77\u6765\u90a3\u662f\u76f8\u5f53\u8f7b\u5feb\u554a~~<\/p>\n
            \u8981\u662f\u6539\u7528Scala\u6216Groovy\u6765\u5b9e\u73b0, \u90a3\u4e48\u8fd8\u8981\u7b80\u5316, \u4f7f\u5f97\u4ee3\u7801\u5473\u66f4\u5c11\u4e9b, \u800c\u4e14\u76f4\u63a5\u8fd0\u884c\u7b80\u5355\u5230\u4e00\u6761\u547d\u4ee4\u5c31\u641e\u5b9a\u4e86:)<\/p>\n
             <\/p>\n","protected":false},"excerpt":{"rendered":"
            \u6700\u8fd1, \u5728\u7f16\u5199\u4e00\u4e2a\u4e13\u95e8\u538b\u6d4bNameNode\u7684\u5de5\u5177(\u4ee5\u4e0b\u7b80\u79f0s4nn), \u5b83\u6709\u4e24\u4e2a\u96be\u70b9 : s4nn\u9700\u8981\u53ef\u4ee5\u6a21\u62df\u4e0a\u4e07\u4e2aDataNode ; s4nn \u9700\u8981\u7075\u6d3b\u7684\u652f\u6301\u5bf9NameNode\u8bbf\u95ee\u884c\u4e3a\u7684\u5b9a\u4e49. \u540e\u8005\u5bfc\u81f4\u4e86\u672c\u6587\u7684\u601d\u8003. \u547d\u4ee4\u884c\u53c2\u6570\u548c\u914d\u7f6e\u6587\u4ef6\u662f\u6700\u5e38\u7528\u6765\u914d\u7f6e\u7cfb\u7edf\u7684\u65b9\u6cd5, \u524d\u8005\u7528\u4e8e\u914d\u7f6e\u9879\u8f83\u5c11, \u540e\u8005\u5219\u9002\u5408\u914d\u7f6e\u590d\u6742\u60c5\u51b5. \u8fd9\u4e24\u79cd\u65b9\u5f0f\u90fd\u6709\u5171\u540c\u4ee4\u4eba\u75db\u82e6\u7684\u5730\u65b9: \u7f16\u5199\u4ee3\u7801\u53bb\u8f7d\u5165->\u89e3\u6790->\u8f6c\u6362, \u901a\u5e38\u5982\u540c\u5904\u7406\u534f\u8bae\u822c\u65e0\u804a(\u8981\u662f\u6709\u4e2a\u4ec0\u4e48\u53d8\u66f4, KMN!!); \u5bf9\u4e8e\u590d\u6742\u7684\u914d\u7f6e\u6587\u4ef6\u7f16\u5199\u800c\u8a00, \u603b\u662f\u6ca1\u6709\u987a\u624b\u7684\u7f16\u8f91\u5668\u652f\u6301, \u5199\u8d77\u6765\u65e2\u7d2f\u53c8\u6613\u9519. \u8981\u662f\u7528\u4ee3\u7801\u53d6\u4ee3\u914d\u7f6e\u6587\u4ef6\u5462? \u5443\u2026 \u8fd9\u4f1a\u5f88\u9ebb\u70e6\u5427, \u50cfjava\u8fd9\u6837\u7684\u6539\u4e86\u4ee3\u7801\u90a3\u8fd8\u4e0d\u8981\u91cd\u65b0\u7f16\u8bd1\u554a? \u55ef, \u786e\u5b9e, \u4f46\u5e76\u6ca1\u6709\u60f3\u8c61\u4e2d\u90a3\u4e48\u9ebb\u70e6, \u4e00\u4e9b\u6280\u5de7\u53ef\u4ee5\u8ba9\u5b83\u53d8\u5f97\u7b80\u5355(\u81f3\u5c11java\u662f\u8fd9\u6837), \u5982Btrace. \u7528\u4ee3\u7801\u53d6\u4ee3\u72ec\u7acb\u7684\u914d\u7f6e\u6587\u4ef6\u4e0d\u662f\u65b0\u9c9c\u7684\u505a\u6cd5, \u50cfGuice,\u00a0Gant \u7b49\u5df2\u7ecf\u4ee5Internal DSL\u7684\u4ee3\u7801\u4ee3\u66ff\u4e86XML. \u597d\u5904\u5f88\u660e\u663e: \u826f\u597d\u7684DSL\u98ce\u683c, \u7b80\u6d01\u6613\u61c2 ; \u514d\u53bb\u5bf9\u914d\u7f6e\u6587\u4ef6\u7684\u89e3\u6790\u8f6c\u6362 ; \u6700\u597d\u7684\u7f16\u8f91\u5668\u652f\u6301, \u8bed\u6cd5\u9ad8\u4eae, \u4e00\u952e\u683c\u5f0f\u5316, \u63d0\u793a\u8865\u5168, \u91cd\u6784 ; \u7f16\u8bd1\u5668\u5e2e\u52a9\u67e5\u9519; \u4e0e\u4ee3\u7801\u65e0\u7f1d\u7ed3\u5408, \u80fd\u591f\u5bb9\u6613\u5728\u53d8\u5316\u4e2d\u4fdd\u6301\u4e00\u81f4\u6027. \u7b80\u8a00\u4e4b\u4e24\u4e2a\u5b57, \u201c\u9ad8\u6548\u201d! \u8fd9\u5012\u662f\u633a\u9002\u5408s4nn\u7684, \u4e0d\u59a8\u4e00\u8bd5! \u4ece\u9700\u6c42\u7684\u89d2\u5ea6\u51fa\u53d1, \u914d\u7f6e\u5e94\u8be5\u80fd\u591f\u5b8c\u6210: \u5b9a\u4e49\u4e00\u7ec4Client RPC\u8c03\u7528\u884c\u4e3a, … Continue reading \u8ba9\u4ee3\u7801\u53d6\u4ee3\u4f60\u7684\u914d\u7f6e\u6587\u4ef6\u5427<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-638","post","type-post","status-publish","format-standard","hentry","category-java"],"_links":{"self":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/638","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=638"}],"version-history":[{"count":2,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/638\/revisions"}],"predecessor-version":[{"id":640,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/638\/revisions\/640"}],"wp:attachment":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=638"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=638"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=638"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}