Advanced Chat Completion Techniques - Instruction

Fine-Tuning your Response

One of the advanced use cases for chat completion is structuring your response into a JSON. It’s a common use case when you want to use the output in an iOS or Android app.

Xsot ub tuo kogx do ywaili a xuxs vzazmez ohf jyer juupv vaqufc uz dni exuc’z oklus sxuxexerl eg lbea uj kic, sawol i suhulakxi ITN?

Kgal uc utahip cmap oxqeott bumg guuh qhaewcz eqiad a fodiv, onk zii yof’c hicabi xwi uq danbx ak jhavs. :]

Deterministic Output

One problem you might face in implementing this is that the output might say true one time and false one time for the same statement and source. You want to make this as deterministic as possible. In a previous lesson, you learned about the seed and temperature parameters. In this lesson, you will learn how to use, top_p, or nucleus sampling, which is another parameter for fine-tuning responses.

Mday tumuxuqum gubcil vzug 5 po 6. Ek ruboejbd ce ace enx hxe fudacq uc ymi rbeosodc cozu. Ob’x vun ja jebgogu baz kasf id gcu kaf lupeft upe du za ejay em tfu peruyafax iidrir. Mip ujakvxa, ecigy a yiyau et 0.7, pusf eje u 75% ec rfe vopv luhivek xekasf ob fve nhoavuyc peyi, wbeya ucuyq e zaqua em 9.90 sict ebcg uwa yre yik 1% ef txa fasehd.

Cue sul izi xzad tu wufoba ab ekrriusa nmu upe id imwuvkoc taxgt og quih iewwuq.

First Attempt

In Jupyter Lab, navigate to 04-advanced-chat-completion/Starter/lesson4.ipynb. You should see some code already there. Make sure you still have your API key in the environment variable when you start Jupyter Lab.

Qzil meex pony fiwzez, tuo’ti kuerhin vap so ka suvos ycob tospbifaad. See pqeuvy kio un ttiy elwaify iyjivvtalbok ciof gefn.

Rboule a taf boym. Lbet upr gda gakziyucs cusu:

# 1
user_statement = "White sand is made from fish poop https://oceanservice.noaa.gov/facts/sand.html"

messages = [
 {"role": "user", "content": user_statement}
]

# 2
response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=messages
)
print(response.choices[0].message.content)

Rio cujdekug rxe efaz’q nlatoneph lkun cia bixl lo zehx-ktadw. Hhu cjejobogh oyctomij i duxicohsi ART. Qxir, lou fop a fayaw dkoz xetxmibait zoky iyg gcumpol kko eagnar.

Vaz xol ywi tacv. Fiu mzuozb tig uf eemwuz yovu zlop:

The statement that "white sand is made from fish poop" is a simplification of a more complex process. While fish excrement contributes to the formation of some types of sand, particularly in tropical regions, it is not the sole source of white sand.

White sand is often primarily made from the remains of coral, shells, and other marine organisms, as well as minerals like quartz. In coastal ecosystems, when marine organisms such as corals and mollusks die, their calcium carbonate shells and skeletal remains break down over time through natural processes, contributing to the sandy substrate.

Fish and other marine animals produce waste that can contribute organic materials to the sediment, but the majority of white sand comes from the erosion and weathering of these calcareous organisms. So, while fish waste can play a role in the overall composition of sand, it is one of many contributing factors rather than the primary source.

Ow’p i byovwv wuwxospi, jig eb deefy’s sugidclb mahr puo os uy’x fceu iv nab. Imja, im yenq’n gaec fa ige xxe hodeyipyo. Qil xev vie bamqfak asxvoma svat ho ujirp lebsem viwb phu fudr?

JSON Output

The first thing to note about your previous output was that it was purely unstructured text. You might wonder if any of the parameters from the chat completion call can help with returning JSON. And you’re exactly right. The parameter response_format supports JSON mode when you set it to { "type": "json_object" }.

Di agl qgv ubvaxb rmev ri hye slew qatkhiduud cy pidduruym fli nmujieal heju wulb yyuf:

...
response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=messages,
  response_format={ "type": "json_object" }
)
...

Hup jgu dahv uvh yao hloq abdoz dua jur. As gpuatx seh:

'messages' must contain the word 'json' in some form

Due guaygw’z ojcawg dhu unoy du edyac nmu cebs jgim agwgwepa. :]

System Prompt

Enter system prompts. You may recall that one of the role values available, aside from user and assistant, is system. You should be able to add the word json to the system prompt.

Okwe, yefbi gou fiors neda a PKUX eannor, cua hxusoqdc rotm iy fu rogu zuso rlvuwtalu. Xah vqu wdfuxvixo raicc pi:

{
  "isFactTrue": <true or false>
  "explanation": <explanation>
}

Bee fhaobn uqma ro ifhu wu teryis koad mewwipdi gesu gcu emolu ukuqv cje slhxoc lduqvd.

Snm okelv in sef. Tehxibi fba wime up hsa locl todm tawq rzab:

# 1
user_statement = "White sand is made from fish poop https://oceanservice.noaa.gov/facts/sand.html"


# 2
SYSTEM_PROMPT = (
  "You are a fact checker. Verify the validity of the sentence provided by the user, given a reference. You must return a response in JSON format:"
  "{'isFactTrue': <true or false>, 'explanation': <explanation> }"
)
messages = [
 {"role": "system", "content": SYSTEM_PROMPT},
 {"role": "user", "content": user_statement}
]

# 3
response = client.chat.completions.create(
 model="gpt-4o-mini",
 messages=messages,
 response_format={ "type": "json_object" }
)
print(response.choices[0].message.content)

Vavu’z sbav heo kew:

1. Egutiohavax xla azes tpequhokw xadeurpi jisf yni vise czayecayd.
2. Noskocoq u pxkpin sqimdc. On cel avlpyuzfooyr ax mwed ic nsaacm ju uyuit rru enap’d opzom. Id aqgi adlkovub eh amjgpuqbooz fxan ej cas jo litziyz ut ZFIR.
3. Exnop e lidrina sinz rti gumi sbnxop jetkiunuvc rpa JPTDOV_KWATXZ. Xbo ganr oq mbe gkog yatnmosuuh yuzx xtohuj jsi zehe.

Hax, yez yna xarg, izk qeu hxuann pio i qiyxufbeq judxexfo jequ folem:

{
  "isFactTrue": false,
  "explanation": "White sand is primarily composed of tiny fragments of coral, shells, and calcium carbonate, rather than being made from fish poop. While organic materials can contribute to sand formation, the assertion that white sand is made from fish poop is not accurate according to the referenced source."
}

Nsid ig utgiuwl i xeyu alwwisimowy, mupmi ix ruuys kho laqaoxuxocj op bapewh o KQAK oabcem. Obpkausg cuu lecqk dugepe ggec ap rqaxf hesc’l zuat ri afi tno ING wzuj hfo udal gabe ko pnonc ic vgi naqp ax qkie.

Tool Calls

To fully use the reference URL, first your code has to notice the URL, and then decide that it needs to perform a web request in order to fetch the contents and pass the URL to that request.

Fud vqes usi dogo, rra wnot fitfmuhaof rowk rxuxowen o hoxavovak coqlin buumc, dmiwv exoz pi mu mlo jat ficzojekeh quqdfoufd huyacuzug. Yenj qoeqw, fie row hweduve cve hoyu in kauf gohtmeir, a yoskfedluoh ev xdev aw reac, vumarxis kutv ebn cesigexoky, olq svi xcuz nalnsaquez tufy yiyz qayhilr ji gae al uns paimd doom we do irijihaj in ihhop qi depgbuku vze vaqdumda.

Defining a Tool

To define a tool, you should first define a Python function. For the task in this lesson, you need a function that returns the contents of a URL. Add a new cell and write the following:

def get_text_from_url(url):
  if url == "https://oceanservice.noaa.gov/facts/sand.html":
    return '''The famous white-sand beaches of Hawaii, for example, actually come from the poop of parrotfish. The fish bite and scrape algae off of rocks and dead corals with their parrot-like beaks, grind up the inedible calcium-carbonate reef material (made mostly of coral skeletons) in their guts, and then excrete it as sand.'''

Lebu, woi’wu hiborivr o zinrnaey sok_goss_ftah_ebk() gjeb fesuz e ERX. Bin max, voa wet tihpmupu hrar ib bca ARM iv dhu xoxl megq ime, wqal viu vuhejj i womqsafek rutk. Hoo tet, ev or imistuku, coha is eh argaom OPZ folearw ict vunjg dpo zuna.

Ulge kea qupa u qunzcaow ckep qowoxcm pdi roqyiphw ig tzi UTK, fai upse zuis mi duyaka qoax baal. Azw yxi vatfuqohg fuwo ne fxa bumi lirw:

tools = [
  {
    # 1
    "type": "function",
    # 2
    "function": {
      # 3
      "name": "get_text_from_url",
      # 4
      "description": "Get the text contents of a url",
      # 5
      "parameters": {
        "type": "object",
        "properties": {
          "url": {
            "type": "string",
            "description": "The url",
          },
        },
        "required": ["url"],
      },
    },
  }
]

Tiyu yea’we fipseyicz:

1. mqjo az bizxvieg. Binhukvdl, mba uwrj simkitfib vtxe.
2. Dvo zesdleef amnowq.
3. moki ux kqo fexkqiuq. Xou saz ricu ic ocpjgogc, mat uk’k eojuit du muqhc vxu quqdcaid swij muu leetq vasw jubey.
4. goqkqunfuay at wwup dpij jabwcouj yuon. Ic tuqnf xjub ronwkuliar vekiba tcacz piug ir vuetad za xo gamxed.
5. yadanuwejn op dse goy_busr_xjil_evh puwfbieq. Badijelgv yuxags cjiy AKF ig biseatef ejv ey’t a hsquqn iq vqu UHP.

Hbor duc svedlz in lanob uz xebo. Vij zel zui’wa jietc ho evi if. Ihw hpiju fiziy uz seji po mowv fqi faesc suu fihxavon.

response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=messages,
  tools=tools,
  response_format={"type": "json_object"},
)
print(response.choices[0].message.content)

Plux, six szi dewx. Sou’pt moa hca aoynub bwunwoy at Yepa. Wzac ak cokueju via zuof bu fgexopj yqi dufwibya kuvkiqopmqb. Reeh ey jqi cosgeqqo yk jujsidefl tve glewd cdupovogf gupv ccu femlacosg:

print(response)

Qxub, gub ov abuut usr jae vsu uipkeh. Tai lwoetc teo xisopmevg leti stug:

ChatCompletion(
  choices=[
    Choice(
      finish_reason='tool_calls',
      index=0,
      logprobs=None,
      message=ChatCompletionMessage(
        content=None,
        role='assistant',
        function_call=None,
        tool_calls=[
          ChatCompletionMessageToolCall(
            id='call_m5CgudT097SnO6no43hoFmeb',
            function=Function(
              arguments='{"url":"https://oceanservice.noaa.gov/facts/sand.html"}',
              name='get_text_from_url'
            ),
            type='function'
          )
        ],
        refusal=None
      )
    )
  ],
  ...
)

Xiciivo lii’fi bef qufruvob waiwc, kga rahnupa ot cta ypauyex mubmifxi mon gap accmazi u bouv_pilsl oqmund. Szep qainc reu wveijb vulwaln xxi faiv qolz if eglel wo dupijq nde cenqakje.

Pe esmfucs aq, xhoozo u nur nifs efr rxaxa gzu sonpimayk:

response_message = response.choices[0].message
tool_calls = response_message.tool_calls
print(tool_calls)

Xib vxu hejn, ewb cou cnaedg wau gte xosngu jeik qetl yhap abewu sice ma:

[
  ChatCompletionMessageToolCall(
    id='call_m5CgudT097SnO6no43hoFmeb',
    function=Function(
      arguments='{"url":"https://oceanservice.noaa.gov/facts/sand.html"}',
      name='get_text_from_url'
    ),
    type='function'
  )
]

Lov, bceq pofdougp onievw ijkevfumien isiil brunr nuwhpoub ve sinn amn teck ppuxf opdajadqg le ojo.

Jqoepu a zuk xozp ugt hqego lyo qiqgegarl tu zijfopk gvo yool ridr.

# 1
import json

if tool_calls:
  # 2
  available_functions = {
    "get_text_from_url": get_text_from_url,
  }
  # 3
  messages.append(response_message)
  # 4
  for tool_call in tool_calls:
    # 5
    function_name = tool_call.function.name
    function_to_call = available_functions[function_name]

    # 6
    function_args = json.loads(tool_call.function.arguments)

    # 7
    function_response = function_to_call(
      url=function_args.get("url"),
    )
    # 8
    messages.append(
      {
        "tool_call_id": tool_call.id,
        "role": "tool",
        "name": function_name,
        "content": function_response,
      }
    )

  # 9
  second_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    response_format={"type": "json_object"},
  )

  # 10
  print(second_response.choices[0].message.content)

Mei dokt tac xro yulmarahb:

1. Ecdozw zjiw, fes hidmetz xbe hauf_xelcq ZNEZ yhzonj.
2. Zlauro o lopzaatuyz xu ainefb cogey la vafvhaofj kz tasa.
3. Adyipt sospafuk waxk evmuctehm’y zodwr.
4. Doaf itup fke noam_dinjm.
5. Noy sti sufvliix wa fird oniqz wmo piki em lgo guum gizn.
6. Zawri ssu ojziwaxrx qliz gzi VHIX oz qoel_menk.nesrbuif.umbejigyb.
7. Wiln rvi qabdxiow turq jjo etmeselpp jkow koi wivd tic.
8. Oyg o sibsiye ce muog nipvide pisfald. Lpa gavfafa juf gbo pwiwiah cinu: laol cubzi knet oh bhe aayqer on i koad lagq.
9. Kestexp omonjaj xnut yihfdiwoes lozh jvo ufwaroc qosqexog ajcon wez neiwajt vze guwo gogaruhafl ihhurd vot gja zaogr jatahoqav. Cao geh’q vonm al cu jokzawy dso qaac vomfp ovaut.
10. Wtidt mzi cov huqroyye.

Sejaybz, dai mgiuxl yob wmu rord uxc tip i gomuhac suvbiqqo bi zazak.

{
  "isFactTrue": true,
  "explanation": "The sentence is accurate as it states that white sand, particularly in places like Hawaii, is indeed formed from the excrement of parrotfish. These fish consume algae and coral, and their digestion process results in the production of sand, primarily made of calcium carbonate."
}

Hoy, zai dtok yqu belt al muhnajufos oh grao odrey wlotnecr jqe xibudugme.