{"id":136187,"date":"2024-09-18T13:02:54","date_gmt":"2024-09-18T07:32:54","guid":{"rendered":"https:\/\/www.vskills.in\/certification\/tutorial\/?page_id=136187"},"modified":"2024-09-18T13:02:55","modified_gmt":"2024-09-18T07:32:55","slug":"loading-documents-and-generating-embeddings-for-chroma-database","status":"publish","type":"page","link":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/","title":{"rendered":"Loading Documents and Generating Embeddings for Chroma Database"},"content":{"rendered":"\n<p>Chroma, a powerful vector database, requires data to be represented as numerical vectors for efficient storage and retrieval. In this comprehensive guide, we will explore the steps involved in loading documents into Chroma and generating their corresponding embeddings.<\/p>\n\n\n\n<p><strong>Prerequisites<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Chroma:<\/strong> Ensure you have Chroma installed on your system.<\/li>\n\n\n\n<li><strong>Embedding Model:<\/strong> Choose a suitable embedding model, such as SentenceTransformer, to generate embeddings for your documents.<\/li>\n\n\n\n<li><strong>Data:<\/strong> Prepare your documents in a suitable format, such as a list of strings or a text file.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Loading Documents<\/strong><\/h2>\n\n\n\n<p><strong>Create a Chroma Client:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>import chromadb<br>client = chromadb.Client()<\/p>\n\n\n\n<p><strong>Create a Collection:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>collection = client.create_collection(<br>name=&#8221;my_collection&#8221;,<br>embedding_function=SentenceTransformer(&#8220;all-MiniLM-L6-v2&#8221;)<br>)<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Generating Embeddings<\/strong><\/h2>\n\n\n\n<p><strong>Import Necessary Libraries:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>from sentence_transformers import SentenceTransformer<\/p>\n\n\n\n<p><strong>Load Documents:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>documents = [&#8220;This is a sample document.&#8221;, &#8220;Another sample document.&#8221;]\n\n\n\n<p><strong>Create Embedding Model:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>model = SentenceTransformer(&#8220;all-MiniLM-L6-v2&#8221;)<\/p>\n\n\n\n<p><strong>Generate Embeddings:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>embeddings = model.encode(documents)<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Adding Documents and Embeddings to Chroma<\/strong><\/h2>\n\n\n\n<p><strong>Add Documents:<\/strong><\/p>\n\n\n\n<p>Python<\/p>\n\n\n\n<p>collection.add(<br>documents=documents,<br>embeddings=embeddings<br>)<\/p>\n\n\n\n<p>By following these steps, you can effectively load documents into a Chroma database and generate their corresponding embeddings. These embeddings will be stored in the database, allowing you to perform similarity search and other operations.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Chroma, a powerful vector database, requires data to be represented as numerical vectors for efficient storage and retrieval. In this comprehensive guide, we will explore the steps involved in loading documents into Chroma and generating their corresponding embeddings. Prerequisites Loading Documents Create a Chroma Client: Python import chromadbclient = chromadb.Client() Create a Collection: Python collection&#8230;<\/p>\n","protected":false},"author":16,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-136187","page","type-page","status-publish","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Loading Documents and Generating Embeddings for Chroma Database - Tutorial<\/title>\n<meta name=\"description\" content=\"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Loading Documents and Generating Embeddings for Chroma Database - Tutorial\" \/>\n<meta property=\"og:description\" content=\"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/\" \/>\n<meta property=\"og:site_name\" content=\"Tutorial\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/vskills.in\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-18T07:32:55+00:00\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/\",\"name\":\"Loading Documents and Generating Embeddings for Chroma Database - Tutorial\",\"isPartOf\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#website\"},\"datePublished\":\"2024-09-18T07:32:54+00:00\",\"dateModified\":\"2024-09-18T07:32:55+00:00\",\"description\":\"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Loading Documents and Generating Embeddings for Chroma Database\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#website\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\",\"name\":\"Tutorial\",\"description\":\"Vskills - A initiative in elearning and certification\",\"publisher\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.vskills.in\/certification\/tutorial\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#organization\",\"name\":\"Vskills\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg\",\"contentUrl\":\"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg\",\"width\":73,\"height\":55,\"caption\":\"Vskills\"},\"image\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/vskills.in\/\",\"https:\/\/x.com\/vskills_in\",\"https:\/\/www.linkedin.com\/company-beta\/1371554\/\",\"https:\/\/www.youtube.com\/channel\/UCMWnscxPwRF_PqXo9B7q_Tw\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Loading Documents and Generating Embeddings for Chroma Database - Tutorial","description":"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/","og_locale":"en_US","og_type":"article","og_title":"Loading Documents and Generating Embeddings for Chroma Database - Tutorial","og_description":"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.","og_url":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/","og_site_name":"Tutorial","article_publisher":"https:\/\/www.facebook.com\/vskills.in\/","article_modified_time":"2024-09-18T07:32:55+00:00","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/","url":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/","name":"Loading Documents and Generating Embeddings for Chroma Database - Tutorial","isPartOf":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#website"},"datePublished":"2024-09-18T07:32:54+00:00","dateModified":"2024-09-18T07:32:55+00:00","description":"Learn how to load documents and generate embeddings for the Chroma database, covering the process of transforming text data into vector.","breadcrumb":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/loading-documents-and-generating-embeddings-for-chroma-database\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.vskills.in\/certification\/tutorial\/"},{"@type":"ListItem","position":2,"name":"Loading Documents and Generating Embeddings for Chroma Database"}]},{"@type":"WebSite","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#website","url":"https:\/\/www.vskills.in\/certification\/tutorial\/","name":"Tutorial","description":"Vskills - A initiative in elearning and certification","publisher":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.vskills.in\/certification\/tutorial\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#organization","name":"Vskills","url":"https:\/\/www.vskills.in\/certification\/tutorial\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/","url":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg","contentUrl":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg","width":73,"height":55,"caption":"Vskills"},"image":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/vskills.in\/","https:\/\/x.com\/vskills_in","https:\/\/www.linkedin.com\/company-beta\/1371554\/","https:\/\/www.youtube.com\/channel\/UCMWnscxPwRF_PqXo9B7q_Tw"]}]}},"_links":{"self":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/136187","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/comments?post=136187"}],"version-history":[{"count":1,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/136187\/revisions"}],"predecessor-version":[{"id":136190,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/136187\/revisions\/136190"}],"wp:attachment":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/media?parent=136187"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/categories?post=136187"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/tags?post=136187"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}