02. LangSmith

Menyiapkan LangSmith trace

LangSmith adalah platform untuk pengembangan, pemantauan, dan pengujian aplikasi LLM. Jika Anda memulai proyek atau pelatihan LangChain, sangat disarankan untuk mengatur LangSmith terlebih dahulu sebelum melanjutkan.

Tracing pada LangSmith

Tracing merupakan alat yang kuat untuk memahami cara kerja aplikasi LLM. LangSmith menyediakan fitur pelacakan terbaik di kelasnya, terlepas dari apakah Anda menggunakan LangChain atau tidak.

Pelacakan dapat membantu dalam masalah berikut:

Hasil akhir yang tidak terduga Alasan mengapa agen mengalami looping Alasan mengapa rantai lebih lambat dari yang diharapkan Jumlah token yang digunakan oleh agen di setiap langkah

Tracing tingkat project

Lihat jumlah pengerjaan, tingkat kesalahan, penggunaan token, informasi penagihan, dan lainnya berdasarkan per project.

Apabila Anda mengeklik suatu project, semua proses yang sudah dijalankan akan muncul.

Penelusuran langkah demi langkah terperinci untuk satu kali jalankan

Setelah satu kali dijalankan, ia mencatat secara rinci tidak hanya hasil pencarian dari dokumen yang diambil, tetapi juga input dan output dari GPT. Ini membantu Anda untuk menentukan apakah Anda perlu mengubah algoritme pencarian atau prompt setelah melihat apa yang diambil.

Selain itu, bagian atas menunjukkan berapa lama setiap proses berlangsung (sekitar 30 detik), jumlah token yang digunakan (5.104), dan bahkan jumlah yang dibebankan jika Anda mengarahkan kursor ke sebuah token.

Menggunakan LangSmith Tracing

Menggunakan pelacakan sangat sederhana.

Mendapatkan API Key LangSmith

Kunjungi https://smith.langchain.com/ (opens in a new tab) untuk mendaftar.
Setelah mendaftar, Anda perlu memverifikasi email Anda.
Klik roda gigi kiri (Pengaturan) - kemudian pilih "Personal" - Lalu klik "Create API Key" untuk mendapatkan API Key.

Pada bagian "Description", masukkan deskripsi yang masuk akal bagi Anda dan klik tombol Buat Kunci API untuk membuatnya.

Salin kunci yang dihasilkan dan lanjutkan ke langkah berikutnya.

(PERHATIAN!) Salin kunci yang dibuat di suatu tempat yang aman agar tidak bocor

Mengatur kunci LangSmith di .env

Pertama, masukkan kunci yang dikeluarkan oleh LangSmith dan informasi project Anda dalam file .env.

LANGCHAIN_TRACING_V2: Atur ke "true" untuk memulai pelacakan.
LANGCHAIN_ENDPOINT: https://api.smith.langchain.com (opens in a new tab) Jangan diubah.
LANGCHAIN_API_KEY: Masukkan kunci yang sudah didapatkan pada langkah sebelumnya.
LANGCHAIN_PROJECT: Jika Anda memasukkan nama project, semua run akan dilacak ke grup project tersebut.

Mengaktifkan tracing di Jupyter Notebook atau Kode

Mengaktifkan pelacakan sangat sederhana. Anda hanya perlu mengatur variabel lingkungan.

Impor apa yang Anda tetapkan di file .env.

Python

# pastikan sudah menginstall package python-dotenv
# !pip install python-dotenv
 
from dotenv import load_dotenv
 
load_dotenv()

Jika tracing yang Anda siapkan telah diaktifkan, dan API KEY serta nama proyek telah diatur dengan benar, maka hal ini sudah cukup.

Namun, jika Anda ingin mengubah nama project atau mengubah tracing, Anda dapat melakukannya dengan kode di bawah ini.

Python

import os
 
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com"
os.environ["LANGCHAIN_PROJECT"] = "Nama Project LangChain"
os.environ["LANGCHAIN_API_KEY"] = "Masukkan LangChain API KEY"

Langchain-altero

Package langchain-altero dibuat untuk membuat fitur-fitur yang berhubungan dengan langchain menjadi lebih mudah digunakan. Package ini dibuat berdasarkan package langchain-teddynote

Bagaimana cara menginstall

Install kode (jalankan dari terminal atau Jupyter Notebook)

Terminal

pip install langchain-altero

Collecting langchain-altero
  Downloading langchain_altero-0.0.2-py3-none-any.whl.metadata (3.0 kB)
Collecting langchain (from langchain-altero)
  Downloading langchain-0.2.14-py3-none-any.whl.metadata (7.1 kB)
Collecting kiwipiepy (from langchain-altero)
  Downloading kiwipiepy-0.18.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.0 kB)
Collecting konlpy (from langchain-altero)
  Downloading konlpy-0.6.0-py2.py3-none-any.whl.metadata (1.9 kB)
Collecting rank-bm25 (from langchain-altero)
  Downloading rank_bm25-0.2.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pinecone-client[grpc] (from langchain-altero)
  Downloading pinecone_client-5.0.1-py3-none-any.whl.metadata (19 kB)
Collecting pinecone-text (from langchain-altero)
  Downloading pinecone_text-0.9.0-py3-none-any.whl.metadata (10 kB)
Collecting olefile (from langchain-altero)
  Downloading olefile-0.47-py2.py3-none-any.whl.metadata (9.7 kB)
Collecting pdf2image (from langchain-altero)
  Downloading pdf2image-1.17.0-py3-none-any.whl.metadata (6.2 kB)
Collecting kiwipiepy-model<0.19,>=0.18 (from kiwipiepy->langchain-altero)
  Downloading kiwipiepy_model-0.18.0.tar.gz (34.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.7/34.7 MB 16.6 MB/s eta 0:00:00
etadata (setup.py) ... ent already satisfied: numpy<2 in /usr/local/lib/python3.10/dist-packages (from kiwipiepy->langchain-altero) (1.26.4)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from kiwipiepy->langchain-altero) (4.66.5)
Collecting JPype1>=0.7.0 (from konlpy->langchain-altero)
  Downloading JPype1-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)
Requirement already satisfied: lxml>=4.1.0 in /usr/local/lib/python3.10/dist-packages (from konlpy->langchain-altero) (4.9.4)
Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (6.0.2)
Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (2.0.32)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (3.10.2)
Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (4.0.3)
Collecting langchain-core<0.3.0,>=0.2.32 (from langchain->langchain-altero)
  Downloading langchain_core-0.2.32-py3-none-any.whl.metadata (6.2 kB)
Collecting langchain-text-splitters<0.3.0,>=0.2.0 (from langchain->langchain-altero)
  Downloading langchain_text_splitters-0.2.2-py3-none-any.whl.metadata (2.1 kB)
Collecting langsmith<0.2.0,>=0.1.17 (from langchain->langchain-altero)
  Downloading langsmith-0.1.99-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (2.8.2)
Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain->langchain-altero) (2.32.3)
Collecting tenacity!=8.4.0,<9.0.0,>=8.1.0 (from langchain->langchain-altero)
  Downloading tenacity-8.5.0-py3-none-any.whl.metadata (1.2 kB)
Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from pdf2image->langchain-altero) (9.4.0)
Requirement already satisfied: certifi>=2019.11.17 in /usr/local/lib/python3.10/dist-packages (from pinecone-client[grpc]->langchain-altero) (2024.7.4)
Collecting pinecone-plugin-inference<2.0.0,>=1.0.3 (from pinecone-client[grpc]->langchain-altero)
  Downloading pinecone_plugin_inference-1.0.3-py3-none-any.whl.metadata (2.2 kB)
Collecting pinecone-plugin-interface<0.0.8,>=0.0.7 (from pinecone-client[grpc]->langchain-altero)
  Downloading pinecone_plugin_interface-0.0.7-py3-none-any.whl.metadata (1.2 kB)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.10/dist-packages (from pinecone-client[grpc]->langchain-altero) (4.12.2)
Requirement already satisfied: urllib3>=1.26.0 in /usr/local/lib/python3.10/dist-packages (from pinecone-client[grpc]->langchain-altero) (2.0.7)
Requirement already satisfied: googleapis-common-protos>=1.53.0 in /usr/local/lib/python3.10/dist-packages (from pinecone-client[grpc]->langchain-altero) (1.63.2)
Requirement already satisfied: grpcio>=1.44.0 in /usr/local/lib/python3.10/dist-packages (from pinecone-client[grpc]->langchain-altero) (1.64.1)
Collecting lz4>=3.1.3 (from pinecone-client[grpc]->langchain-altero)
  Downloading lz4-4.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting protobuf<5.0,>=4.25 (from pinecone-client[grpc]->langchain-altero)
  Downloading protobuf-4.25.4-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Collecting protoc-gen-openapiv2<0.0.2,>=0.0.1 (from pinecone-client[grpc]->langchain-altero)
  Downloading protoc_gen_openapiv2-0.0.1-py3-none-any.whl.metadata (1.5 kB)
Collecting mmh3<5.0.0,>=4.1.0 (from pinecone-text->langchain-altero)
  Downloading mmh3-4.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB)
Requirement already satisfied: nltk<4.0.0,>=3.6.5 in /usr/local/lib/python3.10/dist-packages (from pinecone-text->langchain-altero) (3.8.1)
Collecting python-dotenv<2.0.0,>=1.0.1 (from pinecone-text->langchain-altero)
  Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)
Collecting types-requests<3.0.0,>=2.25.0 (from pinecone-text->langchain-altero)
  Downloading types_requests-2.32.0.20240712-py3-none-any.whl.metadata (1.9 kB)
Collecting wget<4.0,>=3.2 (from pinecone-text->langchain-altero)
  Downloading wget-3.2.zip (10 kB)
  Preparing metadata (setup.py) ... ent already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (2.3.5)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (24.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain->langchain-altero) (1.9.4)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from JPype1>=0.7.0->konlpy->langchain-altero) (24.1)
Collecting jsonpatch<2.0,>=1.33 (from langchain-core<0.3.0,>=0.2.32->langchain->langchain-altero)
  Downloading jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)
Collecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain->langchain-altero)
  Downloading orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.4/50.4 kB 3.4 MB/s eta 0:00:00
ent already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.6.5->pinecone-text->langchain-altero) (8.1.7)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.6.5->pinecone-text->langchain-altero) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.6.5->pinecone-text->langchain-altero) (2024.5.15)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain->langchain-altero) (0.7.0)
Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain->langchain-altero) (2.20.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain->langchain-altero) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain->langchain-altero) (3.7)
Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain->langchain-altero) (3.0.3)
Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.32->langchain->langchain-altero)
  Downloading jsonpointer-3.0.0-py2.py3-none-any.whl.metadata (2.3 kB)
Downloading langchain_altero-0.0.2-py3-none-any.whl (19 kB)
Downloading kiwipiepy-0.18.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 59.4 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.4/19.4 MB 51.8 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 997.8/997.8 kB 27.3 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.6/114.6 kB 8.3 MB/s eta 0:00:00
age-1.17.0-py3-none-any.whl (11 kB)
Downloading pinecone_text-0.9.0-py3-none-any.whl (23 kB)
Downloading rank_bm25-0.2.2-py3-none-any.whl (8.6 kB)
Downloading JPype1-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (488 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 488.6/488.6 kB 25.2 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 389.8/389.8 kB 22.5 MB/s eta 0:00:00
ith-0.1.99-py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.4/140.4 kB 8.7 MB/s eta 0:00:00
anylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 44.5 MB/s eta 0:00:00
mh3-4.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (67 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.6/67.6 kB 4.6 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 8.5 MB/s eta 0:00:00
anylinux2014_x86_64.whl (294 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 294.6/294.6 kB 17.4 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 244.8/244.8 kB 15.6 MB/s eta 0:00:00
anylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.9/141.9 kB 10.3 MB/s eta 0:00:00
odel, wget
  Building wheel for kiwipiepy-model (setup.py) ... odel: filename=kiwipiepy_model-0.18.0-py3-none-any.whl size=34843380 sha256=2ba126df45756b51f39a06ebb1ec767ee573d04ac2e620c3bea0f42519ace041
  Stored in directory: /root/.cache/pip/wheels/0d/ea/f6/abb93f89cc196467624828ec9c29150c29a8399a589ba50bef
  Building wheel for wget (setup.py) ... e=wget-3.2-py3-none-any.whl size=9656 sha256=763e485d3ce3f4a507033f254bfedd946fcee2a1ef4221ae955323dab4474ef5
  Stored in directory: /root/.cache/pip/wheels/8b/f1/7f/5c94f0a7a505ca1c81cd1d9208ae2064675d97582078e6c769
Successfully built kiwipiepy-model wget
Installing collected packages: wget, mmh3, kiwipiepy-model, types-requests, tenacity, rank-bm25, python-dotenv, protobuf, pinecone-plugin-interface, pdf2image, orjson, olefile, lz4, kiwipiepy, jsonpointer, JPype1, pinecone-text, pinecone-plugin-inference, konlpy, jsonpatch, protoc-gen-openapiv2, pinecone-client, langsmith, langchain-core, langchain-text-splitters, langchain, langchain-altero
  Attempting uninstall: tenacity
    Found existing installation: tenacity 9.0.0
    Uninstalling tenacity-9.0.0:
      Successfully uninstalled tenacity-9.0.0
  Attempting uninstall: protobuf
    Found existing installation: protobuf 3.20.3
    Uninstalling protobuf-3.20.3:
      Successfully uninstalled protobuf-3.20.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-metadata 1.15.0 requires protobuf<4.21,>=3.20.3; python_version < "3.11", but you have protobuf 4.25.4 which is incompatible.
Successfully installed JPype1-1.5.0 jsonpatch-1.33 jsonpointer-3.0.0 kiwipiepy-0.18.0 kiwipiepy-model-0.18.0 konlpy-0.6.0 langchain-0.2.14 langchain-altero-0.0.2 langchain-core-0.2.32 langchain-text-splitters-0.2.2 langsmith-0.1.99 lz4-4.3.3 mmh3-4.1.0 olefile-0.47 orjson-3.10.7 pdf2image-1.17.0 pinecone-client-5.0.1 pinecone-plugin-inference-1.0.3 pinecone-plugin-interface-0.0.7 pinecone-text-0.9.0 protobuf-4.25.4 protoc-gen-openapiv2-0.0.1 python-dotenv-1.0.1 rank-bm25-0.2.2 tenacity-8.5.0 types-requests-2.32.0.20240712 wget-3.2

Menyiapkan LangSmith Tracing

Kunci API LangSmith harus diatur dalam file .env (LANGCHAIN_API_KEY)

Python

from langchain_altero import logging
 
# Masukkan nama project.
logging.langsmith("nama project yang diinginkan")

Contoh output

Jika Anda tidak ingin menggunakan tracing, Anda dapat menonaktifkannya sebagai berikut:

Python

from langchain_altero import logging
 
# set_enable=False untuk menonaktifkan logging.
logging.langsmith("Langchain Tutorial Project", set_enable=False)

01. OpenAI API 03. menggunakan openai api gpt-4o multimodal