{"id":58,"count":2,"description":"Transformer\u306f\u3001\u751f\u6210AI\u3084\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\uff08LLM\uff09\u306e\u4e2d\u5fc3\u3068\u306a\u308b\u6df1\u5c64\u5b66\u7fd2\u30e2\u30c7\u30eb\u306e\u69cb\u9020\u3067\u3001\u7279\u306b\u300c\u9577\u3044\u6587\u8108\u3092\u7406\u89e3\u3057\u306a\u304c\u3089\u8a00\u8a9e\u3092\u51e6\u7406\u3059\u308b\u300d\u70b9\u306b\u512a\u308c\u3066\u3044\u307e\u3059\u30022017\u5e74\u306b\u63d0\u6848\u3055\u308c\u305f\u30e2\u30c7\u30eb\u3067\u3001\u5f93\u6765\u4e3b\u6d41\u3060\u3063\u305fRNN\u3084LSTM\u306e\u3088\u3046\u306a\u300c\u9806\u756a\u3069\u304a\u308a\u306b\u8aad\u3080\u4ed5\u7d44\u307f\u300d\u306b\u4f9d\u5b58\u3057\u306a\u3044\u3053\u3068\u304c\u9769\u65b0\u3067\u3057\u305f\u3002\r\n\r\n\u6700\u5927\u306e\u7279\u5fb4\u306f Self-Attention\uff08\u81ea\u5df1\u6ce8\u610f\u6a5f\u69cb\uff09\u3067\u3059\u3002\u3053\u308c\u306f\u3001\u5165\u529b\u3055\u308c\u305f\u6587\u7ae0\u306e\u5404\u5358\u8a9e\u304c\u3001\u6587\u4e2d\u306e\u4ed6\u306e\u3069\u306e\u5358\u8a9e\u306b\u3069\u308c\u3060\u3051\u6ce8\u610f\u3092\u5411\u3051\u308b\u3079\u304d\u304b\u3092\u6570\u5024\u5316\u3059\u308b\u4ed5\u7d44\u307f\u3067\u3059\u3002\u305f\u3068\u3048\u3070\u300c\u5f7c\u306f\u30ea\u30f3\u30b4\u3092\u98df\u3079\u305f\u3002\u305d\u306e\u5f8c\u3001\u5f7c\u306f\u6b69\u3044\u305f\u3002\u300d\u3067\u306f\u30012\u3064\u76ee\u306e\u300c\u5f7c\u300d\u304c\u6587\u8108\u7684\u306b1\u3064\u76ee\u306e\u300c\u5f7c\u300d\u3092\u6307\u3059\u3068\u5224\u65ad\u3067\u304d\u308b\u3088\u3046\u306b\u3001\u6587\u4e2d\u306e\u4f9d\u5b58\u95a2\u4fc2\u3092\u81ea\u52d5\u7684\u306b\u6349\u3048\u307e\u3059\u3002\r\n\r\nTransformer\u306f\u3053\u306eSelf-Attention\u3092\u591a\u982d\uff08Multi-Head\uff09\u3067\u540c\u6642\u306b\u884c\u3044\u3001\u6587\u6cd5\u30fb\u610f\u5473\u30fb\u4f4d\u7f6e\u95a2\u4fc2\u306a\u3069\u3092\u591a\u89d2\u7684\u306b\u628a\u63e1\u3057\u307e\u3059\u3002\u3055\u3089\u306b\u3001\u5165\u529b\u6587\u5168\u4f53\u3092\u4e00\u5ea6\u306b\u51e6\u7406\u3067\u304d\u308b\u305f\u3081\u3001\u4e26\u5217\u8a08\u7b97\u304c\u53ef\u80fd\u3067\u5b66\u7fd2\u52b9\u7387\u304c\u98db\u8e8d\u7684\u306b\u5411\u4e0a\u3057\u307e\u3057\u305f\u3002\r\n\r\n\u57fa\u672c\u69cb\u9020\u306f Encoder\uff08\u5165\u529b\u306e\u7406\u89e3\uff09 \u3068 Decoder\uff08\u51fa\u529b\u306e\u751f\u6210\uff09 \u3067\u69cb\u6210\u3055\u308c\u307e\u3059\u304c\u3001GPT\u306e\u3088\u3046\u306a\u30e2\u30c7\u30eb\u306f\u4e3b\u306bDecoder\u90e8\u5206\u3092\u7528\u3044\u3001\u6b21\u306b\u6765\u308b\u5358\u8a9e\u3092\u4e88\u6e2c\u3059\u308b\u5f62\u3067\u6587\u7ae0\u3092\u751f\u6210\u3057\u307e\u3059\u3002\r\n\r\n\u307e\u3068\u3081\u308b\u3068\u3001Transformer\u306f\r\n\u300c\u81ea\u5df1\u6ce8\u610f\u306b\u3088\u3063\u3066\u6587\u8108\u3092\u6df1\u304f\u7406\u89e3\u3057\u3001\u4e26\u5217\u51e6\u7406\u3067\u9ad8\u901f\u306b\u5b66\u7fd2\u30fb\u751f\u6210\u3067\u304d\u308bAI\u30e2\u30c7\u30eb\u300d\r\n\u3067\u3042\u308a\u3001\u751f\u6210AI\u306e\u6027\u80fd\u3092\u6c7a\u5b9a\u3065\u3051\u308b\u57fa\u76e4\u6280\u8853\u3067\u3059\u3002\r\n","link":"https:\/\/itsenior.jp\/km\/category\/ai-mechanism\/transformer\/","name":"Transformer\u306e\u4ed5\u7d44\u307f","slug":"transformer","taxonomy":"category","parent":47,"meta":[],"_links":{"self":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/categories\/58","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/categories"}],"about":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/taxonomies\/category"}],"up":[{"embeddable":true,"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/categories\/47"}],"wp:post_type":[{"href":"https:\/\/itsenior.jp\/km\/wp-json\/wp\/v2\/posts?categories=58"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}