{"id":10173,"date":"2024-04-12T18:34:36","date_gmt":"2024-04-12T09:34:36","guid":{"rendered":"https:\/\/developers.agirobots.com\/jp\/?p=10173"},"modified":"2024-04-22T23:58:20","modified_gmt":"2024-04-22T14:58:20","slug":"exploring-the-intelligence-of-transformers","status":"publish","type":"post","link":"https:\/\/developers.agirobots.com\/jp\/exploring-the-intelligence-of-transformers\/","title":{"rendered":"\u3010\u89e3\u8aac\u3011ChatGPT\u3067\u3082\u4f7f\u308f\u308c\u308bTransformer\u304c\u8ce2\u3044\u7406\u7531\u306b\u8feb\u308b"},"content":{"rendered":"\n<p>\u7686\u3055\u3093\u3053\u3093\u306b\u3061\u306f\u3002 <\/p>\n\n\n\n<p>\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001\u300c\u3010\u89e3\u8aac\u3011ChatGPT\u3067\u3082\u4f7f\u308f\u308c\u308bTransformer\u304c\u8ce2\u3044\u7406\u7531\u306b\u8feb\u308b\u300d\u3068\u3044\u3046\u30bf\u30a4\u30c8\u30eb\u3067\u3001\u5b9f\u306fTransformer\u306f\u975e\u5e38\u306b\u8133\u306b\u4f3c\u3066\u304a\u308a\u3001\u3072\u3087\u3063\u3068\u3059\u308b\u3068\u601d\u8003\u529b\u3092\u6301\u3063\u3066\u3044\u308b\u306e\u304b\u3082\u3057\u308c\u306a\u3044\u3001\u3068\u3044\u3046\u8a71\u3092\u3057\u3066\u3044\u304d\u305f\u3044\u3068\u601d\u3044\u307e\u3059\uff08\u5f8c\u534a\u306f\u79c1\u306e\u5984\u60f3\u3067\u3059\uff09\u3002<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"\u3010\u89e3\u8aac\u3011ChatGPT\u3067\u3082\u4f7f\u308f\u308c\u308bTransformer\u304c\u8ce2\u3044\u7406\u7531\u306b\u8feb\u308b\" width=\"974\" height=\"548\" src=\"https:\/\/www.youtube.com\/embed\/cxPW24PgWlE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">\u672c\u8a18\u4e8b\u306e\u30e2\u30c1\u30d9\u30fc\u30b7\u30e7\u30f3<\/h2>\n\n\n\n<p>\u3067\u306f\u307e\u305a\u6700\u521d\u306b\u3001\u672c\u8a18\u4e8b\u306e\u30e2\u30c1\u30d9\u30fc\u30b7\u30e7\u30f3\u306b\u3064\u3044\u3066\u304a\u8a71\u3057\u3057\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002\u3064\u3044\u5148\u65e5\u30012024\u5e743\u670827\u65e5\u306b\u300cThe Topos of Transformer Networks\u300d\u3068\u3044\u3046\u8ad6\u6587\u304carXiv\u306b\u6295\u7a3f\u3055\u308c\u3001\u305d\u308c\u304c\u5c11\u3057\u8a71\u984c\u306b\u306a\u308a\u307e\u3057\u305f\u3002\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001ChatGPT\u3092\u306f\u3058\u3081\u3068\u3057\u3066\u5e83\u304f\u4f7f\u7528\u3055\u308c\u3066\u3044\u308bTransformer\u304c\u3001\u4ed6\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3001\u4f8b\u3048\u3070MLP\u3084CNN\u3001RNN\u306a\u3069\u3068\u6bd4\u8f03\u3057\u3066\u3001\u8868\u73fe\u529b\u304c\u5727\u5012\u7684\u306b\u8c4a\u304b\u3067\u3042\u308b\u3053\u3068\u3092\u6570\u5b66\u7684\u306b\u8a3c\u660e\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u8a18\u4e8b\u3067\u306f\u305d\u306e\u8a73\u7d30\u3092\u6271\u3046\u308f\u3051\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u304c\u3001\u3053\u306e\u52d5\u753b\u304c\u6295\u7a3f\u3055\u308c\u305f\u3053\u3068\u3092\u304d\u3063\u304b\u3051\u306b\u3001Transformer\u3068\u3044\u3046\u9ad8\u6027\u80fd\u306a\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u306b\u3064\u3044\u3066\u3082\u3046\u4e00\u5ea6\u81ea\u5206\u306a\u308a\u306b\u8003\u3048\u3066\u307f\u305f\u3044\u3068\u601d\u3063\u305f\u6b21\u7b2c\u3067\u3059\u3002\u8272\u3005\u306a\u8ad6\u6587\u3092\u8abf\u67fb\u3057\u306a\u304c\u3089\u8003\u3048\u305f\u7d50\u679c\u3001Transformer\u306f\u6211\u3005\u306e\u8133\u306b\u304b\u306a\u308a\u4f3c\u3066\u3044\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u611f\u3058\u307e\u3057\u305f\u306e\u3067\u3001\u3053\u306e\u8a18\u4e8b\u3067\u6271\u3044\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Transformer\u306e\u30e2\u30c1\u30d9\u30fc\u30b7\u30e7\u30f3<\/h2>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u307e\u305a\u3001Transformer\u306e\u6982\u8981\u304b\u3089\u8aac\u660e\u3057\u3066\u3044\u304d\u307e\u3059\u3002Transformer\u306f\u300cAttention Is All You Need\u300d\u3068\u547c\u3070\u308c\u308b\u8ad6\u6587\u3067\u63d0\u6848\u3055\u308c\u305f\u624b\u6cd5\u3067\u3001\u56f3\u306b\u793a\u3059\u3088\u3046\u306a\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3092\u3068\u3063\u3066\u3044\u307e\u3059\u3002\u3053\u306eTransformer\u306f\u3001\u5f93\u6765\u306e\u7cfb\u5217\u5909\u63db\u30e2\u30c7\u30eb\u304c\u62b1\u3048\u308b\u4f4d\u7f6e\u4f9d\u5b58\u6027\u3078\u306e\u5f31\u3055\u3068\u3001\u5b66\u7fd2\u901f\u5ea6\u306e\u9045\u3055\u3068\u3044\u3046\u8ab2\u984c\u306b\u5bfe\u51e6\u3057\u305f\u3082\u306e\u3067\u3059\u3002Transformer\u306e\u57fa\u672c\u30d6\u30ed\u30c3\u30af\u306f\u300cTransformer Block\u300d\u3068\u547c\u3070\u308c\u3001\u3053\u308c\u304c\u8907\u6570\u30b9\u30bf\u30c3\u30af\u3055\u308c\u3066\u30a8\u30f3\u30b3\u30fc\u30c0\u30fc\u30fb\u30c7\u30b3\u30fc\u30c0\u30fc\u30e2\u30c7\u30eb\u3092\u69cb\u6210\u3057\u3066\u3044\u307e\u3059\u3002\u305d\u3057\u3066\u3001\u3053\u306eTransformer\u306e\u300cMulti-Head Attention\u300d\u306f\u3001\u8907\u6570\u306e\u300cScaled Dot-Product Attention\u300d\u304c\u4e26\u5217\u306b\u914d\u7f6e\u3055\u308c\u305f\u69cb\u9020\u3092\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u300cScaled Dot-Product Attention\u300d\u306f\u3001\u56f3\u306e1\u756a\u5de6\u306b\u793a\u3055\u308c\u308b\u3088\u3046\u306a\u69cb\u9020\u3068\u306a\u3063\u3066\u304a\u308a\u3001Transformer\u306b\u304a\u3044\u3066\u975e\u5e38\u306b\u91cd\u8981\u306a\u5f79\u5272\u3092\u62c5\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"382\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7-1024x382.png\" alt=\"\" class=\"wp-image-10176\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7-1024x382.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7-300x112.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7-768x287.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7-1536x573.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-7.png 1894w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Transformer\u306b\u3064\u3044\u3066\u306f\u4ee5\u4e0b\u306e\u8a18\u4e8b\u3067\u8a73\u3057\u304f\u89e3\u8aac\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-wp-embed is-provider-agirobots-blog wp-block-embed-agirobots-blog\"><div class=\"wp-block-embed__wrapper\">\n<div><a href=\"https:\/\/developers.agirobots.com\/jp\/attention-mechanism-transformer\/\" class=\"st-cardlink st-embed-cardlink\"><div class=\"kanren st-cardbox\"><dl class=\"clearfix\"><dt class=\"st-card-img\"><img decoding=\"async\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2022\/04\/0BEBE537-0128-44BE-B0BB-932B12A5680B-150x150.jpeg\" alt=\"\" width=\"100\" height=\"100\" \/><\/dt><dd><p class=\"st-cardbox-t\">\u3010\u8a73\u8aac\u3011Attention\u6a5f\u69cb\u306e\u8d77\u6e90\u304b\u3089\u5b66\u3076Transformer<\/p><div class=\"st-card-excerpt smanone\"><p>\u307f\u306a\u3055\u3093\u306f\u3001Transformer\u306b\u3064\u3044\u3066\u3069\u306e\u3088\u3046\u306a\u30a4\u30e1\u30fc\u30b8\u3092\u6301\u3063\u3066\u3044\u307e\u3059\u304b\uff1f\u6700\u8fd1\u306f\u3001BERT\u3001GPT\u306a\u3069\u306eTransformer\u30d9\u30fc\u30b9\u306e\u30e2\u30c7\u30eb\u304c\u76ee\u3092\u898b\u5f35\u308b\u3088\u3046\u306a\u6210\u679c\u3092\u3060\u3057\u3066\u3044\u308b\u306e\u3067\u3001\u305d\u308c\u3089\u3092\u60f3\u50cf\u3059\u308b ... <\/p><\/div><\/dd><\/dl><\/div><\/a><\/div>\n<\/div><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Scaled Dot-Product Attention<\/h3>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u3001Scaled Dot-Product Attention\u306e\u5f0f\u3092\u898b\u3066\u307f\u307e\u3057\u3087\u3046\u3002Scaled Dot-Product Attention\u306e\u5165\u529b\u306b\u306f\u3001\u300cQuery\u300d\u3001\u300cKey\u300d\u3001\u300cValue\u300d\u306e3\u3064\u304c\u3042\u308a\u307e\u3059\u3002\u307e\u305a\u3001\u5de6\u5074\u306e\u56f3\u3092\u898b\u3066\u304f\u3060\u3055\u3044\u3002\u6700\u521d\u306eMatMul\u3067\u306f\u3001Query\u3068Key\u304c\u4e57\u7b97\u3055\u308c\u307e\u3059\u3002\u305d\u306e\u5f8c\u3001\\(\\sqrt{d_k}\\)\u200b\u200b\u3067\u30b9\u30b1\u30fc\u30eb\u5316\u3055\u308c\u3001\u305d\u306e\u7d50\u679c\u306b\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9\u95a2\u6570\u304c\u9069\u7528\u3055\u308c\u307e\u3059\u3002\u6700\u5f8c\u306b\u3001\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9\u95a2\u6570\u304c\u9069\u7528\u3055\u308c\u305f\u884c\u5217\u3068Value\u304cMatMul\u306b\u3088\u3063\u3066\u639b\u3051\u5408\u308f\u305b\u3089\u308c\u307e\u3059\u3002 \u6b21\u306b\u53f3\u306e\u56f3\u3092\u898b\u306a\u304c\u3089\u5b9f\u969b\u306b\u30c8\u30fc\u30af\u30f3\u30d9\u30af\u30c8\u30eb\u3092\u30a4\u30e1\u30fc\u30b8\u3057\u306a\u304c\u3089\u3001\u3069\u306e\u3088\u3046\u306b\u51e6\u7406\u304c\u884c\u308f\u308c\u308b\u306e\u304b\u3092\u898b\u3066\u307f\u307e\u3057\u3087\u3046\u3002\u3042\u308b\u5358\u8a9e\u306e\u30c8\u30fc\u30af\u30f3\u30d9\u30af\u30c8\u30eb\u304c\u56f3\u306e\u3088\u3046\u306a\u884c\u30d9\u30af\u30c8\u30eb\u3067\u8868\u3055\u308c\u308b\u3068\u4eee\u5b9a\u3057\u307e\u3059\u3002\u3053\u306e\u4f8b\u3067\u3001Q\u306f\u3053\u306e1\u500b\u306e\u30c8\u30fc\u30af\u30f3\u30d9\u30af\u30c8\u30eb\u3067\u3059\u3002K\u306b\u3064\u3044\u3066\u306f\u3001\u30c8\u30fc\u30af\u30f3\u30d9\u30af\u30c8\u30eb\u304c5\u3064\u4e26\u3079\u3089\u308c\u305f\u884c\u5217\u3067\u3001\u305d\u308c\u304c\u8ee2\u7f6e\u3055\u308c\u3066\u3044\u307e\u3059\u3002V\u306b\u3064\u3044\u3066\u306fK\u3068\u540c\u3058\u3067\u3059\u3002\u3064\u307e\u308a\u3001\u3053\u308c\u306fSourceTarget Attention\u3067\u3059\u3002\u3053\u306e\u6642\u3001\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9\u306e\u5185\u90e8\u3067\u306f\u3001\u63d0\u4f9b\u3055\u308c\u305fQuery\u3068Key\u3068\u3057\u3066\u4e0e\u3048\u3089\u308c\u305f\u884c\u5217\u306e\u5404\u30c8\u30fc\u30af\u30f3\u306b\u5bfe\u3057\u3066\u5185\u7a4d\u304c\u884c\u308f\u308c\u307e\u3059\u3002\u305d\u3057\u3066\u3001\\(\\sqrt{d_k}\\)\u200b\u200b\u3067\u30b9\u30b1\u30fc\u30eb\u5316\u3055\u308c\u305f\u5f8c\u306b\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9\u95a2\u6570\u304c\u9069\u7528\u3055\u308c\u308b\u3068\u3001\u3059\u3079\u3066\u306e\u5408\u8a08\u304c1\u306b\u306a\u308b\u3088\u3046\u306a\u884c\u30d9\u30af\u30c8\u30eb\u304c\u751f\u6210\u3055\u308c\u307e\u3059\u3002\u305d\u306e\u884c\u30d9\u30af\u30c8\u30eb\u3068\u30d0\u30ea\u30e5\u30fc\u304c\u639b\u3051\u5408\u308f\u3055\u308c\u3001\u65b0\u3057\u3044\u30d9\u30af\u30c8\u30eb\u304c\u8a08\u7b97\u3055\u308c\u307e\u3059\u3002\u3053\u3053\u3067\u306e\u51fa\u529b\u30b5\u30a4\u30ba\u306f\u30af\u30a8\u30ea\u30fc\u3068\u540c\u3058\u3067\u3059\u3002Scaled Dot-Product Attention\u3067\u306f\u3001\u3053\u306e\u3088\u3046\u306a\u8a08\u7b97\u304c\u884c\u308f\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"463\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8-1024x463.png\" alt=\"\" class=\"wp-image-10177\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8-1024x463.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8-300x136.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8-768x348.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8-1536x695.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-8.png 1905w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u8a73\u7d30\u306f\u4ee5\u4e0b\u306e\u8a18\u4e8b\u3067\u89e3\u8aac\u3057\u3066\u3044\u307e\u3059\u306e\u3067\u3082\u3057\u3088\u3051\u308c\u3070\u3001\u305d\u3061\u3089\u3092\u3054\u53c2\u7167\u4e0b\u3055\u3044\u3002<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-wp-embed is-provider-agirobots-blog wp-block-embed-agirobots-blog\"><div class=\"wp-block-embed__wrapper\">\n<div><a href=\"https:\/\/developers.agirobots.com\/jp\/essence-of-scaled-dot-product-attention\/\" class=\"st-cardlink st-embed-cardlink\"><div class=\"kanren st-cardbox\"><dl class=\"clearfix\"><dt class=\"st-card-img\"><img decoding=\"async\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/29EE985D-A0D2-46DE-B748-46041FE81FAD-150x150.png\" alt=\"\" width=\"100\" height=\"100\" \/><\/dt><dd><p class=\"st-cardbox-t\">Scaled Dot-Product Attention\u306e\u672c\u5f53\u306e\u51c4\u3055\u3092\u8003\u3048\u308b<\/p><div class=\"st-card-excerpt smanone\"><p>\u8fd1\u5e74\u3001\u6a5f\u68b0\u5b66\u7fd2\u5206\u91ce\u3067\u5927\u304d\u306a\u6210\u679c\u3092\u3042\u3052\u3066\u3044\u308bTransformer[1]\u306b\u304a\u3044\u3066\u3001\u305d\u306e\u4e2d\u6838\u3092\u306a\u3059Scaled Dot-Product Attention[1]\u306b\u306f\u3001\u9769\u65b0\u7684\u306a\u4ed5\u7d44\u307f\u304c\u5099\u308f\u3063\u3066\u3044\u307e\u3059\u3002\u305d\u306e\u4ed5 ... <\/p><\/div><\/dd><\/dl><\/div><\/a><\/div>\n<\/div><\/figure>\n\n\n\n<p>Scaled Dot-Product Attention\u306b\u306f\u3001\u5b66\u7fd2\u306b\u3088\u3063\u3066\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3055\u308c\u308b\u91cd\u307f\u8981\u7d20\u304c\u5b58\u5728\u3057\u306a\u3044\u3068\u3044\u3046\u70b9\u304c\u975e\u5e38\u306b\u8208\u5473\u6df1\u3044\u3067\u3059\u3002\u4e0b\u56f3\u3092\u3054\u89a7\u304f\u3060\u3055\u3044\u3002\u3053\u3053\u3067\u306f\u3001Query\u3068Key\u3092\u5165\u529b\u3068\u3057\u3001\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9\u3092\u9069\u7528\u3059\u308b\u307e\u3067\u306e\u51e6\u7406\u3092\u884c\u3044\u307e\u3059\u3002\u56f3\u306e\u5de6\u5074\u306b\u306f\u3001Query\u3068Key\u3092\u7528\u3044\u3066\u8a08\u7b97\u3055\u308c\u305f\u884c\u5217\\(\\boldsymbol{W}\\)\u304c\u793a\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u305d\u3057\u3066\u6700\u5f8c\u306eMatMul\u306f\u3001\u884c\u5217\\(\\boldsymbol{W}\\)\u3092\u91cd\u307f\u3068\u3057\u3066\u7528\u3044\u3001Value\u3092\u5165\u529b\u3068\u3059\u308b\u5bc6\u7d50\u5408\u5c64\u3068\u89e3\u91c8\u3067\u304d\u307e\u3059\u3002\u3053\u3053\u3067\u3001\u884c\u5217\\(\\boldsymbol{W}\\)\u306fQuery\u3068Key\u306e\u5185\u7a4d\u306b\u3088\u3063\u3066\u6c7a\u307e\u308b\u305f\u3081\u3001\u5165\u529b\u306b\u3088\u3063\u3066\u52d5\u7684\u306b\u5909\u5316\u3059\u308b\u3053\u3068\u306b\u306a\u308a\u307e\u3059\u3002\u3064\u307e\u308a\u3001Scaled Dot-Product Attention\u306f\u5b66\u7fd2\u306b\u3088\u308a\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3055\u308c\u308b\u91cd\u307f\u306f\u6301\u305f\u306a\u3044\u4e00\u65b9\u3067\u3001\u5165\u529b\u306b\u4f9d\u5b58\u3057\u3066\u52d5\u7684\u306b\u5909\u5316\u3059\u308b\u91cd\u307f\u3092\u3082\u3064\u5bc6\u7d50\u5408\u5c64\u3068\u3057\u3066\u6a5f\u80fd\u3057\u307e\u3059\u3002\u901a\u5e38\u3001\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306f\u5b66\u7fd2\u6642\u306b\u91cd\u307f\u3092\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3057\u3001\u305d\u306e\u5f8c\u306f\u91cd\u307f\u3092\u5909\u66f4\u3057\u306a\u3044\u3053\u3068\u304c\u4e00\u822c\u7684\u3067\u3059\u304c\u3001\u3053\u306eScaled Dot-Product Attention\u306f\u7570\u306a\u308a\u307e\u3059\u3002\u5165\u529b\u30d1\u30bf\u30fc\u30f3\u304c\u7121\u9650\u306b\u3042\u308b\u5834\u5408\u3001\u3053\u306e\u884c\u5217\u306e\u30d1\u30bf\u30fc\u30f3\u3082\u5b9f\u8cea\u7684\u306b\u7121\u9650\u306b\u306a\u308b\u305f\u3081\u3001\u8868\u73fe\u529b\u304c\u975e\u5e38\u306b\u9ad8\u3044\u3068\u8a00\u3048\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"494\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-1024x494.png\" alt=\"\" class=\"wp-image-8441\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-1024x494.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-300x145.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-768x371.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-1536x741.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-2-2048x989.png 2048w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Transformer\u3092\u69cb\u6210\u3059\u308b\u5404\u5c64\u306e\u91cd\u307f\u306e\u6027\u8cea<\/h3>\n\n\n\n<p>\u3053\u3053\u3067\u3001Transformer\u3092\u69cb\u6210\u3059\u308b\u5bc6\u7d50\u5408\u5c64\u306e\u91cd\u307f\u306e\u6027\u8cea\u306b\u3064\u3044\u3066\u7c21\u5358\u306b\u78ba\u8a8d\u3057\u3066\u304a\u304d\u307e\u3057\u3087\u3046\u3002\u4e0b\u56f3\u306e\u53f3\u306b\u306fTransformer Block\u304c\u3001\u771f\u3093\u4e2d\u306b\u306fMulti-Head Attention\u3092\u62e1\u5927\u3057\u305f\u3082\u306e\u304c\u8a18\u8f09\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u8d64\u3067\u56f2\u3063\u305f\u90e8\u5206\u304cScaled Dot-Product Attention\u3067\u3001\u5148\u307b\u3069\u8aac\u660e\u3057\u305f\u901a\u308a\u3001\u91cd\u307f\u304c\u5165\u529b\u7cfb\u5217\u306b\u3088\u3063\u3066\u52d5\u7684\u306b\u5909\u5316\u3059\u308b\u90e8\u5206\u3067\u3059\u3002Transformer\u306b\u306f\u305d\u308c\u4ee5\u5916\u306b\u3082\u591a\u304f\u306e\u5bc6\u7d50\u5408\u5c64\u304c\u3042\u308a\u307e\u3059\u3002\u5177\u4f53\u7684\u306b\u306f\u3001Scaled Dot-Product Attention\u306e\u76f4\u524d\u306eLinear\u5c64\u3001\u305d\u308c\u3089\u3092concat\u3057\u305f\u5f8c\u306eLinear\u5c64\u3001\u305d\u3057\u3066Feed Forward\u5c64\u3067\u3059\uff08\u9752\u3067\u56f2\u3063\u305f\u90e8\u5206\uff09\u3002\u3053\u308c\u3089\u306e\u5c64\u306f\u5b66\u7fd2\u6642\u4ee5\u5916\u3067\u306f\u91cd\u307f\u304c\u5909\u5316\u3057\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<p>RNN\u3084CNN\u306a\u3069\u3001\u5f93\u6765\u306e\u7cfb\u5217\u51e6\u7406\u30e2\u30c7\u30eb\u306f\u5b66\u7fd2\u6642\u4ee5\u5916\u306f\u91cd\u307f\u304c\u56fa\u5b9a\u3055\u308c\u3066\u3044\u308b\u5c64\u306e\u307f\u3067\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3057\u305f\u304c\u3001Transformer\u306f\u5165\u529b\u306b\u3088\u3063\u3066\u91cd\u307f\u304c\u52d5\u7684\u306b\u5909\u308f\u308b\u5c64\u304c\u542b\u307e\u308c\u308b\u69cb\u9020\u306b\u306a\u3063\u3066\u3044\u308b\u305f\u3081\u975e\u5e38\u306b\u9769\u65b0\u7684\u3060\u3068\u8003\u3048\u3089\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"411\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-9-1024x411.png\" alt=\"\" class=\"wp-image-10178\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-9-1024x411.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-9-300x120.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-9-768x308.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-9.png 1374w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">\u6d77\u99ac\u3068\u9023\u60f3\u8a18\u61b6\u30e2\u30c7\u30eb<\/h2>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u3001\u6b21\u306b\u6d77\u99ac\u3068\u9023\u60f3\u8a18\u61b6\u306e\u30e2\u30c7\u30eb\u306b\u3064\u3044\u3066\u8aac\u660e\u3092\u3057\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u5927\u8133\u8fba\u7e01\u7cfb\u3068\u6d77\u99ac<\/h3>\n\n\n\n<p>\u307e\u305a\u3001\u57fa\u790e\u77e5\u8b58\u3068\u3057\u3066\u5927\u8133\u8fba\u7e01\u7cfb\u3068\u6d77\u99ac\u306b\u3064\u3044\u3066\u7c21\u5358\u306b\u8aac\u660e\u3057\u307e\u3059\u3002\u5927\u8133\u8fba\u7e01\u7cfb\u306f\u3001\u6d77\u99ac\u3001\u504f\u6843\u4f53\u3001\u5e2f\u72b6\u56de\u3068\u3044\u3063\u305f\u8907\u6570\u306e\u69cb\u9020\u304b\u3089\u6210\u308a\u7acb\u3063\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u30b7\u30b9\u30c6\u30e0\u306f\u57fa\u672c\u7684\u306a\u611f\u60c5\u306e\u51e6\u7406\u3084\u30b9\u30c8\u30ec\u30b9\u53cd\u5fdc\u3001\u8a18\u61b6\u306e\u5f62\u6210\u3068\u4fdd\u5b58\u3001\u9577\u671f\u8a18\u61b6\u3078\u306e\u8ee2\u9001\u306a\u3069\u975e\u5e38\u306b\u91cd\u8981\u306a\u5f79\u5272\u3092\u679c\u305f\u3057\u3066\u3044\u307e\u3059\u3002\u305d\u306e\u4e2d\u3067\u3082\u6d77\u99ac\u306f\u3001\u77ed\u671f\u8a18\u61b6\u3092\u9577\u671f\u8a18\u61b6\u3078\u3068\u5909\u63db\u3059\u308b\u30d7\u30ed\u30bb\u30b9\u306b\u4e0d\u53ef\u6b20\u3067\u3042\u308a\u3001\u9069\u5207\u306b\u8a18\u61b6\u3092\u4fdd\u6301\u3059\u308b\u306a\u3069\u601d\u8003\u306b\u5927\u304d\u304f\u95a2\u4e0e\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"795\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-10-1024x795.png\" alt=\"\" class=\"wp-image-10179\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-10-1024x795.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-10-300x233.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-10-768x597.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-10.png 1030w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">\u6d77\u99ac\u306e\u5185\u90e8\u69cb\u9020\u3068\u6570\u7406\u30e2\u30c7\u30eb<\/h3>\n\n\n\n<p>\u3067\u306f\u3001\u6d77\u99ac\u306e\u5185\u90e8\u69cb\u9020\u3068\u6570\u7406\u30e2\u30c7\u30eb\u306b\u3064\u3044\u3066\u8aac\u660e\u3057\u307e\u3059\u3002\u6d77\u99ac\u306fCA1\u3001CA2\u3001CA3\u3068\u547c\u3070\u308c\u308b\u9818\u57df\u306b\u5206\u3051\u3089\u308c\u3066\u304a\u308a\u3001\u305d\u308c\u305e\u308c\u304c\u7279\u6709\u306e\u6a5f\u80fd\u3068\u795e\u7d4c\u56de\u8def\u3092\u6301\u3063\u3066\u3044\u307e\u3059\u3002CA1\u306b\u3064\u3044\u3066\u306f\u3001\u4e3b\u306b\u4ed6\u306e\u8133\u9818\u57df\u304b\u3089\u306e\u60c5\u5831\u3092\u53d7\u3051\u53d6\u308a\u3001\u305d\u3057\u3066\u9001\u308a\u51fa\u3059\u3068\u3044\u3046\u64cd\u4f5c\u3092\u884c\u3063\u3066\u3044\u307e\u3059\u3002CA2\u306f\u793e\u4f1a\u7684\u8a18\u61b6\u3084\u884c\u52d5\u306b\u7279\u306b\u95a2\u4e0e\u3057\u3066\u3044\u308b\u3068\u8003\u3048\u3089\u308c\u3066\u3044\u307e\u3059\u3002CA3\u306b\u3064\u3044\u3066\u306f\u3001\u30d1\u30bf\u30fc\u30f3\u306e\u691c\u51fa\u3084\u8a18\u61b6\u306e\u518d\u751f\u3001\u3064\u307e\u308a\u9023\u60f3\u8a18\u61b6\u306b\u5f37\u304f\u95a2\u4e0e\u3057\u3066\u3044\u308b\u3068\u3055\u308c\u3066\u304a\u308a\u3001\u4e0b\u56f3\u306e\u53f3\u306b\u793a\u3059\u3088\u3046\u306a\u76f8\u4e92\u7d50\u5408\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u304c\u898b\u3089\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"326\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11-1024x326.png\" alt=\"\" class=\"wp-image-10180\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11-1024x326.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11-300x96.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11-768x245.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11-1536x489.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-11.png 1604w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">\u9023\u60f3\u8a18\u61b6\u30e2\u30c7\u30eb\u306eHopfield Network<\/h3>\n\n\n\n<p>\u3053\u306e\u76f8\u4e92\u7d50\u5408\u578b\u306e\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306f\u9023\u60f3\u8a18\u61b6\u306e\u30e2\u30c7\u30eb\u3068\u3057\u3066\u91cd\u5b9d\u3055\u308c\u3066\u304a\u308a\u3001\u305d\u306e\u9023\u60f3\u8a18\u61b6\u306e\u30e2\u30c7\u30eb\u306e\u4ee3\u8868\u7684\u306a\u3082\u306e\u306bHopfield Network\u304c\u3042\u308a\u307e\u3059\u3002\u3053\u3053\u3067\u306f\u7c21\u5358\u306bHopfield Network\u306b\u3064\u3044\u3066\u8aac\u660e\u3057\u307e\u3059\u3002Hopfield Network\u3068\u306f\u3001\u4e0b\u56f3\u53f3\u306b\u793a\u3059\u3088\u3046\u306b\u5404\u3005\u306e\u30cb\u30e5\u30fc\u30ed\u30f3\u304c\u76f8\u4e92\u306b\u7d50\u5408\u3059\u308b\u69cb\u9020\u3068\u306a\u3063\u3066\u304a\u308a\u3001\u60c5\u5831\u3092\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306e\u72b6\u614b\u3068\u3057\u3066\u8a18\u61b6\u3057\u307e\u3059\u3002\u521d\u671f\u306eHopfield Network\u3067\u306f\u3001\u7d50\u5408\u306e\u91cd\u307f\u306f\u30d8\u30d6\u306e\u6cd5\u5247\u3092\u7528\u3044\u3066\u6c7a\u5b9a\u3057\u307e\u3059\u3002\u3053\u306eHopfield Network\u3067\u60c5\u5831\u3092\u601d\u3044\u51fa\u3059\u3001\u3064\u307e\u308a\u60f3\u8d77\u3059\u308b\u969b\u306b\u306f\u3001\u521d\u671f\u72b6\u614b\u304b\u3089\u5f90\u3005\u306b\u30a8\u30cd\u30eb\u30ae\u30fc\u95a2\u6570\u306e\u5024\u304c\u5c0f\u3055\u304f\u306a\u308b\u3088\u3046\u306b\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306e\u72b6\u614b\u3092\u66f4\u65b0\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u3053\u306e\u30a8\u30cd\u30eb\u30ae\u30fc\u95a2\u6570\u306b\u304a\u3044\u3066\u6975\u5c0f\u3068\u306a\u3063\u3066\u3044\u308b\u90e8\u5206\u304c\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u304c\u8a18\u61b6\u3057\u3066\u3044\u308b\u72b6\u614b\u3067\u3059\u3002\u3067\u3059\u304b\u3089\u3001\u3069\u3053\u304b\u3089\u59cb\u307e\u3063\u3066\u3082\u3044\u305a\u308c\u3069\u3053\u304b\u306e\u6975\u5c0f\u5024\u306b\u5230\u9054\u3057\u3001\u4f55\u304b\u3057\u3089\u3092\u601d\u3044\u51fa\u3059\u308f\u3051\u3067\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"377\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-12-1024x377.png\" alt=\"\" class=\"wp-image-10181\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-12-1024x377.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-12-300x110.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-12-768x283.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-12.png 1480w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u3001Hopfield Network\u306e\u5177\u4f53\u7684\u306a\u4f7f\u3044\u65b9\u306b\u3064\u3044\u3066\u7c21\u5358\u306b\u8aac\u660e\u3057\u307e\u3059\u3002\u3053\u3053\u3067\u306f\u3001Hopfield Network\u306b\u30ea\u30f3\u30b4\u306e\u753b\u50cf\u3092\u899a\u3048\u3055\u305b\u305f\u3068\u3057\u307e\u3059\u3002\u3053\u3053\u3067\u3001\u5b66\u7fd2\u5f8c\u306eHopfield Network\u306b\u3001\u4e00\u90e8\u304c\u6b20\u3051\u753b\u50cf\u3092\u5165\u529b\u3059\u308b\u3053\u3068\u3092\u8003\u3048\u307e\u3059\u3002\u3053\u306e\u3088\u3046\u306b\u4e0d\u5b8c\u5168\u306a\u753b\u50cf\u304c\u4e0e\u3048\u3089\u308c\u305f\u3068\u3057\u3066\u3082\u3001Hopfield Network\u306f\u72b6\u614b\u3092\u66f4\u65b0\u3059\u308b\u306b\u3064\u308c\u3066\u3001\u6b20\u3051\u3066\u3044\u308b\u90e8\u5206\u3092\u5f90\u3005\u306b\u601d\u3044\u51fa\u3059\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002\u305d\u3057\u3066\u3001\u6700\u7d42\u7684\u306b\u6975\u5c0f\u5024\u306b\u5230\u9054\u3057\u305f\u3068\u3053\u308d\u3067\u5b8c\u5168\u306a\u30ea\u30f3\u30b4\u306e\u753b\u50cf\u3092\u601d\u3044\u51fa\u3057\u307e\u3059\u3002\u3053\u3053\u306b\u793a\u3057\u305f\u30a4\u30e9\u30b9\u30c8\u306f\u3001\u5206\u304b\u308a\u3084\u3059\u304f\u3059\u308b\u305f\u3081\u306b\u6700\u521d\u306b\u899a\u3048\u3055\u305b\u305f\u30ea\u30f3\u30b4\u306e\u753b\u50cf\u306e\u4e00\u90e8\u3092\u6b20\u304b\u305b\u305f\u753b\u50cf\u3092\u5165\u529b\u3057\u3066\u3044\u307e\u3059\u304c\u3001\u5b9f\u969b\u306b\u306f\u3082\u3063\u3068\u7570\u306a\u308b\u753b\u50cf\u3092\u5165\u529b\u3057\u3066\u3082\u9069\u5207\u306b\u601d\u3044\u51fa\u3059\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"500\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-1024x500.png\" alt=\"\" class=\"wp-image-8531\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-1024x500.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-300x147.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-768x375.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-1536x751.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2023\/06\/image-15-2048x1001.png 2048w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Modern Hopfield Network<\/h3>\n\n\n\n<p>\u305f\u3060\u3057\u3001Hopfield Network\u306b\u306f\u8a18\u61b6\u5bb9\u91cf\u304c\u5c11\u306a\u3044\u3068\u3044\u3046\u3053\u3068\u3084\u3001\u5225\u306e\u8a18\u61b6\u3092\u8aa4\u3063\u3066\u60f3\u8d77\u3057\u3066\u3057\u307e\u3063\u305f\u308a\u3001\u8a18\u61b6\u540c\u58eb\u304c\u885d\u7a81\u3057\u3066\u3001\u6700\u521d\u306b\u8a18\u61b6\u3055\u305b\u305f\u8a18\u61b6\u306b\u306f\u5b58\u5728\u3057\u306a\u3044\u306f\u305a\u306e\u8a18\u61b6\u304c\u8aa4\u3063\u3066\u5f15\u304d\u51fa\u3055\u308c\u3066\u3057\u307e\u3046\u3068\u3044\u3046\u73fe\u8c61\u304c\u3042\u308a\u307e\u3057\u305f\u3002\u305d\u306e\u305f\u3081\u30012020\u5e74\u306b\u300eHopfield Network is All You Need\u300f\u3068\u3044\u3046\u65b0\u3057\u3044\u8ad6\u6587\u3067Modern Hopfield Network\u304c\u63d0\u6848\u3055\u308c\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<p>\u3053\u306eModern Hopfield Network\u306f\u3001\u5f93\u6765\u306eHopfield Network\u306e\u8a18\u61b6\u5bb9\u91cf\u306e\u5c11\u306a\u3055\u3092\u4e3b\u306b\u6539\u5584\u3057\u305f\u3082\u306e\u3067\u3001\u65b0\u3057\u3044\u30a8\u30cd\u30eb\u30ae\u30fc\u95a2\u6570\u3068\u66f4\u65b0\u30eb\u30fc\u30eb\u3092\u63d0\u4f9b\u3059\u308b\u3053\u3068\u306b\u3088\u308a\u3001\u6307\u6570\u95a2\u6570\u7684\u306a\u81a8\u5927\u306a\u8a18\u61b6\u5bb9\u91cf\u3092\u5b9f\u73fe\u3057\u3066\u3044\u307e\u3059\u3002\u4ee5\u964d\u306e\u8a71\u3067\u306f\u3001Modern Hopfield Network\u304c\u975e\u5e38\u306b\u91cd\u8981\u306a\u8981\u7d20\u3068\u306a\u3063\u3066\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u9023\u60f3\u8a18\u61b6\u30e2\u30c7\u30eb\u3068Attention<\/h2>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u3001\u9023\u60f3\u8a18\u61b6\u30e2\u30c7\u30eb\u3068Attention\u306b\u3064\u3044\u3066\u305d\u306e\u95a2\u4fc2\u3092\u8aac\u660e\u3057\u3066\u3044\u304d\u307e\u3059\u3002Hopfield Network is All You Need\u3067\u306f\u3001Modern Hopfield Network\u3068Scaled Dot-Product Attention\u306e\u8a08\u7b97\u5f0f\u304c\u975e\u5e38\u306b\u4f3c\u3066\u3044\u308b\u3068\u3044\u3046\u3053\u3068\u304c\u7d39\u4ecb\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" width=\"1024\" height=\"327\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13-1024x327.png\" alt=\"\" class=\"wp-image-10182\" style=\"width:660px;height:auto\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13-1024x327.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13-300x96.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13-768x246.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13-1536x491.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-13.png 1914w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Modern Hopfield Network<\/strong>\u3068<strong>Scaled Dot-Product Attention<\/strong><\/h3>\n\n\n\n<p>\u3053\u308c\u306f\u3069\u3046\u3044\u3046\u3053\u3068\u304b\u3068\u8a00\u3046\u3068\u3001Modern Hopfield Network\u306e\u66f4\u65b0\u5f0f\u304b\u3089Attention\u304c\u5c0e\u51fa\u3067\u304d\u308b\u3068\u3044\u3046\u3053\u3068\u3067\u3059\u3002<\/p>\n\n\n\n<p>\u672c\u8a18\u4e8b\u3067\u306f\u3001Modern Hopfield Network\u306e\u66f4\u65b0\u5f0f\u304b\u3089Scaled Dot-Product Attention\u306e\u8a08\u7b97\u5f0f\u3092\u5c0e\u51fa\u3057\u3066\u307f\u307e\u3057\u3087\u3046\u3002<\/p>\n\n\n\n<p>\u307e\u305a\u3001Modern Hopfield Network\u306e\u66f4\u65b0\u5f0f\u306f\u6b21\u306e\u3088\u3046\u306b\u8868\u73fe\u3055\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\boldsymbol{\\xi}^{new} = \\boldsymbol{X}\\text{softmax}(\\beta  \\boldsymbol{X}^T \\boldsymbol{\\xi})<br>$$<\/p>\n\n\n\n<p>\u3053\u3053\u3067\\( \\boldsymbol{X}\\)\u3068\u306f\u30d9\u30af\u30c8\u30eb\u3067\u8868\u3055\u308c\u305f\u8a18\u61b6\u30d1\u30bf\u30fc\u30f3\u3092\u8907\u6570\u4e26\u3079\u3066\u884c\u5217\u306b\u3057\u305f\u3082\u306e\u3067\u3001 \\(\\boldsymbol{\\xi}\\)\u306f\u30cb\u30e5\u30fc\u30ed\u30f3\u306e\u72b6\u614b\u3092\u8868\u3059\u30d9\u30af\u30c8\u30eb\u3068\u306a\u3063\u3066\u3044\u307e\u3059\u3002\u3067\u306fModern Hopfield Network\u306e\u66f4\u65b0\u5f0f\u306e\u6587\u5b57\u3092\u7f6e\u304d\u63db\u3048\u3066\u3044\u304d\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002\u307e\u305a\u3001\\( \\boldsymbol{\\xi}\\)\u3092\\( \\boldsymbol{Q}^T\\)\u306b\u3001\\( \\boldsymbol{X}\\)\u3092\\( \\boldsymbol{K}^T\\)\u306b\u3001\\( \\boldsymbol{\\beta}\\)\u3092\\( \\sqrt{d_k}\\)\u306b\u7f6e\u304d\u63db\u3048\u308b\u3068\u3001\u3053\u306e\u3088\u3046\u306b\u8868\u3059\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>(\\boldsymbol{Q}^{new})^T = \\boldsymbol{K}^T\\text{softmax}\\left(\\frac{1}{\\sqrt{d_k}}\\boldsymbol{KQ}^T\\right)<br>$$<\/p>\n\n\n\n<p>\u305d\u3057\u3066\u3001\u3053\u306e\u5f0f\u5168\u4f53\u3092\u8ee2\u7f6e\u3059\u308b\u3068\u3001\u6b21\u306e\u3088\u3046\u306b\u66f8\u304d\u63db\u3048\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\boldsymbol{Q}^{new} = \\text{softmax}\\left(\\frac{1}{\\sqrt{d_k}}\\boldsymbol{QK}^T\\right)\\boldsymbol{K}<br>$$<\/p>\n\n\n\n<p>\u3053\u3053\u3067\u3001\u6700\u3082\u53f3\u7aef\u306e\\(\\boldsymbol{K}\\)\u3092\\(\\boldsymbol{V}\\)\u306b\u3001\u305d\u3057\u3066\u65b0\u3057\u3044\\(\\boldsymbol{Q}\\)\uff08\\(=\\boldsymbol{Q}^{new}\\)\uff09\u3092\\(\\boldsymbol{Z}\\)\u306b\u66f8\u304d\u63db\u3048\u308b\u3068\u3001Scaled Dot-Product Attention\u306e\u5f0f\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\boldsymbol{Z} = \\text{softmax}\\left(\\frac{1}{\\sqrt{d_k}}\\boldsymbol{QK}^T\\right)\\boldsymbol{Z}<br>$$<\/p>\n\n\n\n<p>\u305f\u3060\u3057\u3001\u3053\u3053\u3067\u306f\u8aac\u660e\u3092\u7c21\u7565\u5316\u3059\u308b\u305f\u3081\u306b\u53b3\u5bc6\u6027\u306b\u6b20\u3051\u308b\u5f0f\u5909\u5f62\u3092\u3057\u3066\u3044\u308b\u70b9\u306f\u3054\u4e86\u627f\u304f\u3060\u3055\u3044\u3002\u305d\u306e\u305f\u3081\u3001\u6b63\u78ba\u306a\u5f0f\u5909\u5f62\u3092\u77e5\u308a\u305f\u3044\u65b9\u306f\u539f\u8ad6\u6587\u3092\u3054\u53c2\u7167\u304f\u3060\u3055\u3044\u3002<\/p>\n\n\n\n<p>\u3072\u3068\u307e\u305a\u3001Modern Hopfield Network\u306e\u66f4\u65b0\u5f0f\u3068Scaled Dot-Product Attention\u306e\u8a08\u7b97\u5f0f\u304c\u985e\u4f3c\uff08\u4f7f\u3044\u65b9\u306b\u3088\u3063\u3066\u306f\u5168\u304f\u540c\u3058\u306b\u306a\u308b\uff09\u3057\u3066\u3044\u308b\u3053\u3068\u3092\u3054\u7406\u89e3\u3044\u305f\u3060\u3051\u308b\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>Hopfield Network\u306f\u5148\u307b\u3069\u8aac\u660e\u3057\u305f\u901a\u308a\u3001\u6d77\u99ac\u306eCA3\u306e\u30e2\u30c7\u30eb\u3068\u306a\u3063\u3066\u3044\u307e\u3059\u304b\u3089\u3001\u8981\u3059\u308b\u306b\u3053\u3053\u3067\u306fScaled Dot-Product Attention\u306f\u6d77\u99ac\u306eCA3\u306e\u5f79\u5272\u3092\u679c\u305f\u3057\u3066\u3044\u308b\u3068\u89e3\u91c8\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306e\u3067\u306f\u306a\u3044\u3067\u3057\u3087\u3046\u304b\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Transformer\u306f\u8133\u306b\u4f3c\u3066\u3044\u308b\u304b\u3082<\/h2>\n\n\n\n<p>\u305d\u308c\u3067\u306f\u3001Transformer\u304c\u8133\u306b\u4f3c\u3066\u3044\u308b\u3068\u3044\u3046\u8a71\u3092\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u3053\u3053\u304b\u3089\u306f\u79c1\u306e\u5984\u60f3\u304c\u304b\u306a\u308a\u5165\u3063\u3066\u304f\u308b\u305f\u3081\u3001\u8a71\u534a\u5206\u3067\u805e\u3044\u3066\u3044\u305f\u3060\u3051\u308c\u3070\u3068\u601d\u3044\u307e\u3059\u3002Transformer\u3092\u69cb\u6210\u3059\u308b\u57fa\u672c\u30d6\u30ed\u30c3\u30af\u3067\u3042\u308b\u300cTransformer Block\u300d\u306f\u3001\u6d77\u99ac\u3068\u5927\u8133\u3092\u6301\u3064\u3068\u3066\u3082\u8133\u306b\u4f3c\u305f\u69cb\u9020\u3092\u3057\u3066\u3044\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u601d\u3063\u3066\u3044\u307e\u3059\u3002\u4e0b\u56f3\u306b\u793a\u3059\u3088\u3046\u306b\u3001Transformer Block\u3092\u3001\u8907\u6570\u306eScaled Dot-Product Attention\u304c\u4e26\u5217\u306b\u306a\u3063\u3066\u69cb\u6210\u3055\u308c\u305f\u300cMulti-Head Attention\u300d\u3092\u6d77\u99ac\u306b\u898b\u7acb\u3066\u3001\u305d\u306e\u5f8c\u306b\u7d9a\u304fFeed Forward\u3092\u5927\u8133\u306b\u898b\u7acb\u3066\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"642\" height=\"735\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-16.png\" alt=\"\" class=\"wp-image-10185\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-16.png 642w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-16-262x300.png 262w\" sizes=\"(max-width: 642px) 100vw, 642px\" \/><\/figure>\n\n\n\n<p>\u3053\u306eFeed Forward\u306e\u90e8\u5206\u304c\u5927\u8133\u306a\u306e\u304b\u5c0f\u8133\u306a\u306e\u304b\u3001\u5927\u8133\u8fba\u7e01\u7cfb\u306a\u306e\u304b\u3001\u307e\u305f\u306f\u6d77\u99ac\u306e\u4e00\u90e8\u306a\u306e\u304b\u306b\u3064\u3044\u3066\u306f\u7279\u306b\u610f\u898b\u304c\u5206\u304b\u308c\u308b\u3068\u60f3\u50cf\u3057\u307e\u3059\u304c\u3001\u6d77\u99ac\u306f\u5927\u8133\u3068\u975e\u5e38\u306b\u5bc6\u63a5\u306b\u63a5\u7d9a\u3055\u308c\u3066\u3044\u308b\u306e\u3067\u3001\u3053\u3053\u3067\u306f\u5927\u8133\u3068\u3057\u3066\u3044\u307e\u3059\u3002\u305d\u3057\u3066\u3001Transformer\u3067\u306f\u3053\u306eTransformer Block\u304c\u8907\u6570\u30b9\u30bf\u30c3\u30af\u3055\u308c\u308b\u308f\u3051\u3067\u3059\u304b\u3089\u3001Multi-Head Attention\u3067\u9023\u60f3\u3057\u305f\u8a18\u61b6\u3092Feed Forward\u3067\u51e6\u7406\u3057\u3001\u6b21\u306e\u5c64\u306eMulti-Head Attention\u3067\u66f4\u306b\u9023\u60f3\u3059\u308b\u3068\u3044\u3046\u51e6\u7406\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"360\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17-1024x360.png\" alt=\"\" class=\"wp-image-10186\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17-1024x360.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17-300x106.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17-768x270.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17-1536x540.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-17.png 1765w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u3067\u3059\u304b\u3089\u3001Transformer\u306f\u6d77\u99ac\u3068\u5927\u8133\u306e\u9593\u3067\u884c\u3063\u3066\u3044\u308b\u60c5\u5831\u51e6\u7406\u306b\u8fd1\u3044\u51e6\u7406\u3092\u884c\u3063\u3066\u3044\u308b\u3068\u3044\u3046\u89e3\u91c8\u304c\u3067\u304d\u308b\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002\u305d\u3046\u306a\u308b\u3068\u3001Transformer\u306f\u3072\u3087\u3063\u3068\u3059\u308b\u3068\u601d\u8003\u304c\u3067\u304d\u308b\u306e\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"314\" src=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18-1024x314.png\" alt=\"\" class=\"wp-image-10187\" srcset=\"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18-1024x314.png 1024w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18-300x92.png 300w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18-768x236.png 768w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18-1536x471.png 1536w, https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-18.png 1770w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u3053\u308c\u307e\u3067\u3001\u305f\u3068\u3048\u3070\u8996\u899a\u91ce\u3092\u30e2\u30c7\u30eb\u5316\u3057\u305f\u30cd\u30aa\u30b3\u30b0\u30cb\u30c8\u30ed\u30f3\u304cCNN\u306e\u63d0\u6848\u306b\u3064\u306a\u304c\u3063\u305f\u308a\u3001\u5c0f\u8133\u304c\u591a\u5c64\u30d1\u30fc\u30bb\u30d7\u30c8\u30ed\u30f3\u306e\u53ef\u80fd\u6027\u3092\u6301\u3064\u3068\u3055\u308c\u305f\u3053\u3068\u304c\u3042\u308a\u307e\u3059\u3002\u3053\u308c\u3089\u306f\u8133\u306e\u5c40\u6240\u7684\u306a\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u69cb\u9020\u306e\u518d\u73fe\u306b\u7559\u307e\u3063\u3066\u3044\u307e\u3057\u305f\u304c\u3001Transformer\u306f\u6d77\u99ac\u3068\u3057\u3066\u6a5f\u80fd\u3059\u308bAttention\u30e1\u30ab\u30cb\u30ba\u30e0\u3092\u53d6\u308a\u5165\u308c\u308b\u3053\u3068\u3067\u9023\u60f3\u3059\u308b\u80fd\u529b\u3092\u7372\u5f97\u3057\u307e\u3057\u305f\u3002MLP\u3084CNN\u306e\u3088\u3046\u306a\u5f93\u6765\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306f\u3001\u5165\u529b\u306b\u5bfe\u3059\u308b\u51fa\u529b\u3092\u5358\u306b\u5bfe\u5fdc\u4ed8\u3051\u3066\u5b66\u7fd2\u3059\u308b\u3060\u3051\u3067\u3057\u305f\u3002\u4f8b\u3048\u3070\u3001\u3042\u308b\u753b\u50cf\u304c\u5165\u529b\u3055\u308c\u305f\u5834\u5408\u3001\u305d\u308c\u304c\u4f55\u3067\u3042\u308b\u304b\u3092\u8b58\u5225\u3059\u308bCNN\u306f\u3001\u753b\u50cf\u306b\u5bfe\u3059\u308b\u30e9\u30d9\u30eb\u3092\u5b66\u7fd2\u3059\u308b\u3060\u3051\u3067\u3001\u9023\u60f3\u6a5f\u80fd\u306f\u6301\u3063\u3066\u3044\u307e\u305b\u3093\u3067\u3057\u305f\u3002\u307e\u305f\u3001\u6642\u7cfb\u5217\u3092\u6271\u3046RNN\u3001\u7279\u306bLSTM\u306b\u306fCEC\u30bb\u30eb\u3068\u3044\u3046\u8a18\u61b6\u30bb\u30eb\u304c\u542b\u307e\u308c\u3001\u3053\u308c\u306f\u4e00\u898b\u6d77\u99ac\u3068\u985e\u4f3c\u3057\u3066\u3044\u307e\u3059\u304c\u3001CEC\u30bb\u30eb\u306f\u5358\u306a\u308b\u8a18\u61b6\u6a5f\u80fd\u306b\u904e\u304e\u305a\u3001\u9023\u60f3\u3059\u308b\u529b\u306f\u3042\u308a\u307e\u305b\u3093\u3067\u3057\u305f\u3002\u3057\u304b\u3057\u3001Transformer\u306f\u9023\u60f3\u3059\u308b\u529b\u3092\u6301\u3063\u3066\u304a\u308a\u3001\u79c1\u305f\u3061\u304c\u601d\u8003\u3059\u308b\u4e0a\u3067\u91cd\u8981\u306a\u9023\u60f3\u304c\u53ef\u80fd\u3067\u3059\u3002\u3053\u306e\u305f\u3081\u3001Transformer\u304c\u601d\u8003\u3057\u3066\u3044\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\uff08\u601d\u8003\u306e\u5b9a\u7fa9\u306b\u306f\u3088\u308b\u3068\u601d\u3044\u307e\u3059\u304c\uff09\u3002<\/p>\n\n\n\n<p>\u3067\u306f\u3001\u6539\u3081\u3066Transformer\u3068\u5f93\u6765\u306e\u30e2\u30c7\u30eb\u3068\u306e\u9055\u3044\u3092\u8003\u3048\u3066\u307f\u307e\u3057\u3087\u3046\u3002\u5f93\u6765\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306f\u5358\u306a\u308b\u4e38\u6697\u8a18\u578b\u3067\u3059\u304c\u3001Transformer\u306f\u8003\u3048\u306a\u304c\u3089\u7406\u89e3\u3059\u308b\u30bf\u30a4\u30d7\u3067\u3059\u3002\u3064\u307e\u308a\u3001Transformer\u306f\u9023\u60f3\u30d7\u30ed\u30bb\u30b9\u3092\u901a\u3058\u3066\u77e5\u8b58\u3092\u5b66\u7fd2\u3057\u307e\u3059\u3002\u5f93\u6765\u306e\u7cfb\u5217\u30e2\u30c7\u30eb\u304c\u6587\u7ae0\u3092\u30c8\u30fc\u30af\u30f3\u5217\u3068\u3057\u3066\u6271\u3063\u3066\u3044\u305f\u306e\u306b\u5bfe\u3057\u3001Transformer\u306f\u6587\u7ae0\u306e\u5168\u4f53\u50cf\u3092\u72b6\u614b\u3068\u3057\u3066\u4e00\u5ea6\u306b\u6349\u3048\u3066\u5165\u529b\u3059\u308b\u305f\u3081\u3001\u6587\u7ae0\u306e\u72b6\u614b\u3092\u9023\u60f3\u3057\u3001\u672a\u77e5\u306e\u6587\u7ae0\u306b\u3082\u67d4\u8edf\u306b\u5bfe\u5fdc\u3067\u304d\u308b\u3068\u8003\u3048\u3089\u308c\u307e\u3059\u3002\u3053\u308c\u306f\u4e38\u6697\u8a18\u578b\u3067\u306f\u306a\u3044\u304b\u3089\u3053\u305d\u5b9f\u73fe\u3067\u304d\u308b\u3053\u3068\u3067\u3059\u3002\u3082\u3046\u4e00\u5ea6\u8a00\u3044\u307e\u3059\u304c\u3001Transformer\u306f\u601d\u8003\u3057\u3066\u3044\u308b\u306e\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u3055\u3044\u3054\u306b<\/h2>\n\n\n\n<p>\u672c\u8a18\u4e8b\u306b\u306f\u3001Transformer\u304c\u5f93\u6765\u578b\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306b\u6bd4\u3079\u3066\u3001\u305d\u306e\u8868\u73fe\u529b\u304c\u5727\u5012\u7684\u306b\u8c4a\u304b\u3067\u3042\u308b\u3053\u3068\u304c\u6570\u5b66\u7684\u306b\u8a3c\u660e\u3055\u308c\u305f\u3053\u3068\u3092\u5207\u3063\u639b\u3051\u306b\u3001\u79c1\u306a\u308a\u306bTransformer\u306b\u3064\u3044\u3066\u8003\u3048\u305f\u3053\u3068\u3092\u8a18\u8f09\u3057\u307e\u3057\u305f\u3002Attention\u6a5f\u69cb\u306f\u4f1a\u8a71\u306e\u6d41\u308c\u3092\u7406\u89e3\u3057\u3001\u6587\u7ae0\u5168\u4f53\u3092\u4e00\u3064\u306e\u72b6\u614b\u3068\u3057\u3066\u6349\u3048\u3001\u9023\u60f3\u3092\u901a\u3058\u3066\u8a18\u61b6\u304b\u3089\u9069\u5207\u306a\u77e5\u8b58\u3092\u5f15\u304d\u51fa\u3059\u306e\u306b\u91cd\u8981\u306a\u5f79\u5272\u3092\u679c\u305f\u3057\u3066\u3044\u308b\u3068\u8003\u3048\u3089\u308c\u307e\u3059\u3002Transformer\u306e\u69cb\u9020\u306f\u3001\u6d77\u99ac\u3068\u5927\u8133\u3092\u6301\u3064\u8133\u69cb\u9020\u306b\u4f3c\u3066\u304a\u308a\u3001\u3053\u308c\u306b\u3088\u308a\u601d\u8003\u304c\u5fc5\u8981\u306a\u8907\u96d1\u306a\u51e6\u7406\u306b\u3082\u9069\u5fdc\u3067\u304d\u308b\u306e\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<p>\u4ee5\u4e0a\u3067\u672c\u8a18\u4e8b\u306e\u5185\u5bb9\u306f\u4ee5\u4e0a\u3067\u3059\u3002\u5f8c\u307e\u3067\u304a\u8aad\u307f\u3044\u305f\u3060\u304d\u3042\u308a\u304c\u3068\u3046\u3054\u3056\u3044\u307e\u3057\u305f\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u8fd1\u5e74\u3001\u6a5f\u68b0\u5b66\u7fd2\u5206\u91ce\u3067\u5927\u304d\u306a\u6210\u679c\u3092\u3042\u3052\u3066\u3044\u308bTransformer[1]\u306b\u304a\u3044\u3066\u3001\u305d\u306e\u4e2d\u6838\u3092\u306a\u3059Scaled Dot-Product Attention[1]\u306b\u306f\u3001\u9769\u65b0\u7684\u306a\u4ed5\u7d44\u307f\u304c\u5099\u308f\u3063\u3066\u3044\u307e\u3059\u3002\u305d\u306e\u4ed5 &#8230; <\/p>\n","protected":false},"author":1,"featured_media":10174,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_vk_print_noindex":"","sitemap_hide":"","_veu_custom_css":"","veu_display_promotion_alert":"common","vkexunit_cta_each_option":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[170,172,171,106,174,186,206],"tags":[],"class_list":["post-10173","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-ai","category-learn","category-ml","category-dl","category-neuroscience","category-206"],"veu_head_title_object":{"title":"","add_site_title":""},"jetpack_featured_media_url":"https:\/\/developers.agirobots.com\/jp\/wp-content\/uploads\/2024\/04\/image-6.png","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/posts\/10173","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/comments?post=10173"}],"version-history":[{"count":26,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/posts\/10173\/revisions"}],"predecessor-version":[{"id":10226,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/posts\/10173\/revisions\/10226"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/media\/10174"}],"wp:attachment":[{"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/media?parent=10173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/categories?post=10173"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/developers.agirobots.com\/jp\/wp-json\/wp\/v2\/tags?post=10173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}