{"id":23038,"date":"2023-04-21T21:31:53","date_gmt":"2023-04-21T19:31:53","guid":{"rendered":"https:\/\/www.zylstra.org\/blog\/?p=23038"},"modified":"2023-04-21T21:32:03","modified_gmt":"2023-04-21T19:32:03","slug":"how-many-tokens-from-your-blog-are-in-googles-llm","status":"publish","type":"post","link":"https:\/\/www.zylstra.org\/blog\/2023\/04\/how-many-tokens-from-your-blog-are-in-googles-llm\/","title":{"rendered":"How Many Tokens From Your Blog Are In Google&#8217;s LLM?"},"content":{"rendered":"<p><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 384 512\" width=\"12\" height=\"17\"><path d=\"M0 512V48C0 21.49 21.49 0 48 0h288c26.51 0 48 21.49 48 48v464L192 400 0 512z\"\/><\/svg> <em>Bookmarked <a class=\"u-bookmark-of\" href=\"https:\/\/www.washingtonpost.com\/technology\/interactive\/2023\/ai-chatbot-learning\/\">Inside the secret list of websites that make AI like ChatGPT sound smart<\/a> (by By Kevin Schaul, Szu Yu Chen and Nitasha Tiku in the Washington Post)<\/em><\/p>\n<p>The Washington Post takes a closer look at Google&#8217;s C4 dataset, which is comprised of the content of 15 million websites, and has been used to train various LLM&#8217;s. Perhaps also the one used by OpenAI for e.g. ChatGPT, although it&#8217;s not known what OpenAI has been using as source material.<\/p>\n<p>They include a search engine, which let&#8217;s you submit a domain name and find out how many tokens it contributed to the dataset (a token is usually a word, or part of a word).<\/p>\n<p>Obviously I looked at some of the domains I use. This blog is the 102860th contributor to the dataset, with 200.000 tokens (1\/10000% of the total).<\/p>\n<p><a href=\"https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens-640x193.png\" alt=\"\" width=\"640\" height=\"193\" class=\"aligncenter size-large wp-image-23040\" srcset=\"https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens-640x193.png 640w, https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens-300x91.png 300w, https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens-668x202.png 668w, https:\/\/www.zylstra.org\/wp\/wp-content\/uploads\/2023\/04\/blogtokens.png 722w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><br \/><em>Screenshot of the Washington Post&#8217;s search tool, showing the result for this domain, zylstra.org.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Washington Post let&#8217;s you search a domain&#8217;s contribution to Google&#8217;s C4 dataset, used to train various LLM for various #generativeAI #AI<br \/>\nMy blog contributed 200.000 tokens. Your&#8217;s?<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"webmentions_disabled_pings":false,"webmentions_disabled":false,"footnotes":""},"categories":[487,2159,446,86],"tags":[2229,2213,2186],"class_list":["post-23038","post","type-post","status-publish","format-standard","category-timeline","category-eaap","category-linklog","category-metablogging","tag-c4","tag-chatgpt","tag-llm","h-entry","hentry"],"archive_meta":{"categories":["Day to Day","Ethics as a Practice","Linklog","metablogging"],"tags":["c4","chatgpt","llm"],"page_categories":[],"format":null,"kind":null,"ref":null},"_links":{"self":[{"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/posts\/23038","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/comments?post=23038"}],"version-history":[{"count":3,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/posts\/23038\/revisions"}],"predecessor-version":[{"id":23042,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/posts\/23038\/revisions\/23042"}],"wp:attachment":[{"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/media?parent=23038"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/categories?post=23038"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.zylstra.org\/blog\/wp-json\/wp\/v2\/tags?post=23038"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}