{"id":975,"date":"2025-11-14T20:33:48","date_gmt":"2025-11-14T11:33:48","guid":{"rendered":"https:\/\/itsenior.jp\/km\/?p=975"},"modified":"2025-11-14T20:33:48","modified_gmt":"2025-11-14T11:33:48","slug":"transformer","status":"publish","type":"post","link":"https:\/\/itsenior.jp\/km\/transformer\/","title":{"rendered":"Transformer\u3068\u306f\u4f55\u304b\uff1f"},"content":{"rendered":"\n

Transformer\u306f\u30012017\u5e74\u306bGoogle\u304c\u767a\u8868\u3057\u305f\u8ad6\u6587\u300cAttention Is All You Need\u300d\u306b\u3088\u3063\u3066\u63d0\u6848\u3055\u308c\u305f\u6df1\u5c64\u5b66\u7fd2\u30e2\u30c7\u30eb<\/strong>\u3067\u3042\u308a\u3001\u73fe\u5728\u306e\u751f\u6210AI\u3068\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\uff08LLM\uff09\u306e\u57fa\u76e4\u3092\u6210\u3059\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3067\u3059\u3002\u3053\u308c\u4ee5\u524d\u306e\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u3067\u306f\u3001RNN\u3084LSTM\u306e\u3088\u3046\u306a\u300c\u5358\u8a9e\u3092\u9806\u756a\u306b\u51e6\u7406\u3059\u308b\u300d\u4ed5\u7d44\u307f\u304c\u4e3b\u6d41\u3067\u3057\u305f\u304c\u3001Transformer\u306f\u305d\u308c\u3089\u3092\u5b8c\u5168\u306b\u7f6e\u304d\u63db\u3048\u3001\u9ad8\u5ea6\u306a\u6587\u8108\u7406\u89e3\u3001\u9577\u6587\u51e6\u7406\u3001\u5727\u5012\u7684\u306a\u5b66\u7fd2\u52b9\u7387<\/strong>\u3092\u5b9f\u73fe\u3057\u307e\u3057\u305f\u3002<\/p>\n\n\n\n

Transformer\u306e\u4e2d\u5fc3\u7684\u306a\u9769\u65b0\u306f\u3001Attention\uff08\u6ce8\u610f\u6a5f\u69cb\uff09<\/strong>\u3001\u3068\u304f\u306b Self-Attention\uff08\u81ea\u5df1\u6ce8\u610f\uff09<\/strong> \u306b\u3042\u308a\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u6587\u4e2d\u306e\u5358\u8a9e\u304c\u4ed6\u306e\u5358\u8a9e\u3068\u3069\u306e\u3088\u3046\u306b\u95a2\u4fc2\u3057\u3066\u3044\u308b\u304b\u3092\u4e00\u5ea6\u306b\u5b66\u7fd2\u3059\u308b\u4ed5\u7d44\u307f<\/strong>\u3067\u3059\u3002RNN\u306e\u3088\u3046\u306b\u6587\u3092\u9010\u6b21\u7684\u306b\u8aad\u3080\u306e\u3067\u306f\u306a\u304f\u3001Transformer\u306f\u6587\u7ae0\u5168\u4f53\u3092\u300c\u4e00\u6c17\u306b\u300d\u8aad\u307f\u8fbc\u307f\u3001\u6587\u8108\u7684\u306a\u3064\u306a\u304c\u308a\u3092\u540c\u6642\u306b\u628a\u63e1\u3067\u304d\u307e\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u9577\u8ddd\u96e2\u4f9d\u5b58\u95a2\u4fc2<\/strong>\u2500\u2500\u305f\u3068\u3048\u3070\u6587\u982d\u306e\u4e3b\u8a9e\u3068\u6587\u672b\u306e\u8ff0\u8a9e\u306e\u95a2\u4fc2\u2500\u2500\u3092\u6b63\u78ba\u306b\u51e6\u7406\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n

Self-Attention\u306e\u4ed5\u7d44\u307f<\/h3>\n\n\n\n

Self-Attention\u3067\u306f\u3001\u5404\u5358\u8a9e\u304c\u300cQuery\uff08\u8cea\u554f\uff09\u300d\u300cKey\uff08\u9375\uff09\u300d\u300cValue\uff08\u5024\uff09\u300d\u3068\u3044\u30463\u7a2e\u985e\u306e\u30d9\u30af\u30c8\u30eb\u306b\u5909\u63db\u3055\u308c\u307e\u3059\u3002\u30e2\u30c7\u30eb\u306fQuery\u3068Key\u306e\u985e\u4f3c\u5ea6\u3092\u8a08\u7b97\u3057\u3001\u300c\u3069\u306e\u5358\u8a9e\u306b\u3069\u306e\u7a0b\u5ea6\u6ce8\u610f\u3092\u5411\u3051\u308b\u3079\u304d\u304b\u300d\u3092\u6c7a\u5b9a\u3057\u307e\u3059\u3002\u305d\u3057\u3066Value\u3092\u91cd\u307f\u4ed8\u3051\u3057\u3066\u5408\u6210\u3059\u308b\u3053\u3068\u3067\u3001\u6587\u8108\u60c5\u5831\u3092\u53cd\u6620\u3057\u305f\u65b0\u3057\u3044\u5358\u8a9e\u8868\u73fe\u3092\u751f\u6210\u3057\u307e\u3059\u3002<\/p>\n\n\n\n

\u3053\u306e\u4ed5\u7d44\u307f\u306b\u3088\u308a\u3001<\/p>\n\n\n\n

    \n
  • \u6587\u306e\u69cb\u9020<\/li>\n\n\n\n
  • \u5358\u8a9e\u9593\u306e\u610f\u5473\u95a2\u4fc2<\/li>\n\n\n\n
  • \u9577\u6587\u306e\u8ad6\u7406\u7684\u6d41\u308c \u306a\u3069\u3092\u67d4\u8edf\u304b\u3064\u52b9\u7387\u7684\u306b\u5b66\u7fd2\u3067\u304d\u307e\u3059\u3002<\/li>\n<\/ul>\n\n\n\n

    Multi-Head Attention\uff1a\u8907\u6570\u306e\u8996\u70b9\u3092\u540c\u6642\u306b\u6301\u3064<\/h3>\n\n\n\n

    Transformer\u306f\u3053\u308c\u30921\u3064\u3060\u3051\u3067\u306a\u304f\u3001\u8907\u6570\u306eSelf-Attention\u3092\u4e26\u5217\u306b\u8d70\u3089\u305b\u308b Multi-Head Attention<\/strong> \u3092\u63a1\u7528\u3057\u307e\u3059\u3002\u5404\u30d8\u30c3\u30c9\u306f\u8a9e\u5f59\u7684\u95a2\u4fc2\u3001\u6587\u6cd5\u69cb\u9020\u3001\u610f\u5473\u5f79\u5272\u306a\u3069\u7570\u306a\u308b\u89b3\u70b9\u304b\u3089\u6587\u3092\u89e3\u6790\u3057\u3001\u6700\u5f8c\u306b\u7d71\u5408\u3057\u307e\u3059\u3002<\/p>\n\n\n\n

    \u3053\u306e\u591a\u8996\u70b9\u6027<\/strong>\u304c\u3001Transformer\u304c\u300c\u6587\u7ae0\u306e\u8c4a\u304b\u306a\u610f\u5473\u69cb\u9020\u300d\u3092\u628a\u63e1\u3067\u304d\u308b\u7406\u7531\u3067\u3059\u3002<\/p>\n\n\n\n

    Positional Encoding\uff1a\u8a9e\u9806\u3092\u30e2\u30c7\u30eb\u306b\u4f1d\u3048\u308b<\/h3>\n\n\n\n

    Attention\u306f\u8a9e\u9806\u3092\u8003\u616e\u3057\u306a\u3044\u305f\u3081\u3001Transformer\u306f\u300cPositional Encoding\u300d\u3068\u3044\u3046\u4f4d\u7f6e\u60c5\u5831\u3092\u5358\u8a9e\u306b\u52a0\u7b97\u3057\u307e\u3059\u3002\u3053\u308c\u306b\u3088\u308a\u3001\u30e2\u30c7\u30eb\u306f\u5358\u8a9e\u304c\u6587\u306e\u3069\u3053\u306b\u73fe\u308c\u305f\u306e\u304b\u3092\u7406\u89e3\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n

    \u30a8\u30f3\u30b3\u30fc\u30c0\u3068\u30c7\u30b3\u30fc\u30c0<\/h3>\n\n\n\n

    Transformer\u306f\u3082\u3068\u3082\u3068<\/p>\n\n\n\n

      \n
    • \u30a8\u30f3\u30b3\u30fc\u30c0\uff08\u5165\u529b\u306e\u7406\u89e3\uff09<\/strong><\/li>\n\n\n\n
    • \u30c7\u30b3\u30fc\u30c0\uff08\u51fa\u529b\u306e\u751f\u6210\uff09<\/strong> \u306e\uff12\u90e8\u69cb\u9020\u3068\u3057\u3066\u8a2d\u8a08\u3055\u308c\u307e\u3057\u305f\u3002<\/li>\n<\/ul>\n\n\n\n

      \u3057\u304b\u3057LLM\uff08GPT\u306a\u3069\uff09\u306f\u3001\u6587\u7ae0\u751f\u6210\u306b\u7279\u5316\u3057\u305f \u30c7\u30b3\u30fc\u30c0\u306e\u307f\u306e\u69cb\u9020<\/strong> \u3092\u63a1\u7528\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u308c\u306b\u3088\u308a\u300c\u6b21\u306b\u7d9a\u304f\u5358\u8a9e\u3092\u9806\u306b\u751f\u6210\u3059\u308b\u300d\u3068\u3044\u3046\u751f\u6210\u30bf\u30b9\u30af\u306b\u975e\u5e38\u306b\u9069\u3057\u305f\u30e2\u30c7\u30eb\u306b\u306a\u308a\u307e\u3057\u305f\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

      Transformer\u306f\u30012017\u5e74\u306bGoogle\u304c\u767a\u8868\u3057\u305f\u8ad6\u6587\u300cAttention Is All You Need\u300d\u306b\u3088\u3063\u3066\u63d0\u6848\u3055\u308c\u305f\u6df1\u5c64\u5b66\u7fd2\u30e2\u30c7\u30eb\u3067\u3042\u308a\u3001\u73fe\u5728\u306e\u751f\u6210AI\u3068\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\uff08LLM\uff09\u306e\u57fa\u76e4\u3092\u6210\u3059\u30a2\u30fc\u30ad\u30c6 […]<\/p>\n","protected":false},"author":1,"featured_media":976,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"swell_btn_cv_data":"","footnotes":""},"categories":[58,47],"tags":[],"class_list":["post-975","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-transformer","category-ai-mechanism"],"_links":{"self":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/posts\/975","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/comments?post=975"}],"version-history":[{"count":1,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/posts\/975\/revisions"}],"predecessor-version":[{"id":977,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/posts\/975\/revisions\/977"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/media\/976"}],"wp:attachment":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/media?parent=975"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/categories?post=975"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/tags?post=975"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}