Skip to content

Commit

Permalink
Update results
Browse files Browse the repository at this point in the history
  • Loading branch information
capjamesg committed Dec 26, 2023
1 parent 5c47251 commit 460cf23
Show file tree
Hide file tree
Showing 2 changed files with 103 additions and 19 deletions.
32 changes: 13 additions & 19 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ <h1>How's GPT-4 with Vision Doing?</h1>
<p>You can contribute your own tests, too! See the <a href="https://github.com/roboflow/gpt-checkup?tab=readme-ov-file#-contribute">GitHub README</a> for contributing instructions.</p>
</div>
<div class="header_subtitle">
<p>Tests are run every day at 1am PT. Last updated December 25, 2023.</p>
<p>Tests are run every day at 1am PT. Last updated December 26, 2023.</p>
<p>Made with ❤️ by the team at <a href="https://roboflow.com">Roboflow</a>.</p>
</div>
<div class="header_cta">
Expand All @@ -58,12 +58,12 @@ <h1>How's GPT-4 with Vision Doing?</h1>
<div class="feature_header" style="min-height: auto">
<div class="feature_header_text" style="gap: var(--spacing-sizing-4)">
<h2>Response Time</h2>
<p style="font-size: 16px; color: var(--gray-700)">Today, the average response time to receive results from our tests was <b>5.69 seconds</b> per request.</p>
<p style="font-size: 16px; color: var(--gray-700)">Today, the average response time to receive results from our tests was <b>5.58 seconds</b> per request.</p>
<p class="subtitle">This number only accounts for requests made by this application.</p>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>5.69 s</p>
<p>5.58 s</p>
</div>
</div>
</div>
Expand Down Expand Up @@ -176,7 +176,7 @@ <h3><span class="explainer_icon far fa-comment-dots"></span>Prompt</h3>
<h3><span class="explainer_icon far fa-image"></span>Image</h3>
<img class="test_image" src="images/fruit.jpeg" alt="Image of the input into GPT-4" />
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>{'x': 0.375, 'y': 0.3, 'width': 0.25, 'height': 0.4}</pre>
<pre>{'x': 0.64, 'y': 0.35, 'width': 0.16, 'height': 0.3}</pre>
<p class="subtitle" style="margin-top: 16px; text-align: center">Test submitted by <a href="https://roboflow.com/" target="_blank">Roboflow</a></p>
</div>
</div>
Expand Down Expand Up @@ -233,20 +233,20 @@ <h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>```json
{
"A": {
"quantity": 10,
"price": 15
"quantity": 15,
"price": 20
},
"B": {
"quantity": 20,
"quantity": 30,
"price": 25
},
"C": {
"quantity": 30,
"quantity": 40,
"price": 35
},
"D": {
"quantity": 40,
"price": 45
"quantity": 45,
"price": 42
}
}
```</pre>
Expand Down Expand Up @@ -303,13 +303,7 @@ <h3><span class="explainer_icon far fa-comment-dots"></span>Prompt</h3>
<h3><span class="explainer_icon far fa-image"></span>Image</h3>
<img class="test_image" src="images/color.png" alt="Image of the input into GPT-4" />
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>```json
{
"R": 128,
"G": 0,
"B": 128
}
```</pre>
<pre>Failed to produce a valid JSON output: I'm sorry, I am not able to provide RGB color codes for elements in images. If you have any other questions or need assistance with something else, feel free to ask!</pre>
<p class="subtitle" style="margin-top: 16px; text-align: center">Test submitted by <a href="https://roboflow.com/" target="_blank">Roboflow</a></p>
</div>
</div>
Expand Down Expand Up @@ -423,8 +417,8 @@ <h3><span class="explainer_icon far fa-image"></span>Image</h3>
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>```json
{
"length": 2.5,
"width": 2.5
"length": 3.0,
"width": 3.0
}
```</pre>
<p class="subtitle" style="margin-top: 16px; text-align: center">Test submitted by <a href="https://roboflow.com/" target="_blank">Roboflow</a></p>
Expand Down
90 changes: 90 additions & 0 deletions results/2023-12-26.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
{
"zero_shot_classification": {
"score": 1,
"success": true,
"price": 0.00481,
"pass_fail": "Pass",
"response_time": 6.003724575042725,
"result": "Toyota Camry"
},
"count_fruit": {
"score": 0,
"success": false,
"price": 0.007870000000000002,
"pass_fail": "Fail",
"response_time": 4.22366189956665,
"result": "9"
},
"document_ocr": {
"score": 1,
"success": true,
"price": 0.00857,
"pass_fail": "Pass",
"response_time": 2.577056407928467,
"result": "I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times."
},
"handwriting_ocr": {
"score": 1,
"success": true,
"price": 0.008730000000000002,
"pass_fail": "Pass",
"response_time": 5.760297060012817,
"result": "The words of songs on the album have been echoing in my head all week. \"Fades into the grey of my day old tea.\""
},
"extraction_ocr": {
"score": 1.0,
"success": true,
"price": 0.00725,
"pass_fail": "Pass",
"response_time": 4.001994609832764,
"result": "[{'name': 'MARY THOMAS', 'time_per_day': 1, 'medication': 'ATENOLOL', 'dosage': 100, 'rx_number': '1234567-12345'}]"
},
"math_ocr": {
"score": 1.0,
"success": true,
"price": 0.01789,
"pass_fail": "Pass",
"response_time": 6.463160991668701,
"result": "3x^2-6x+2"
},
"object_detection": {
"score": 0.14790018259281795,
"success": false,
"price": 0.009490000000000002,
"pass_fail": "Fail",
"response_time": 3.284456253051758,
"result": "{'x': 0.64, 'y': 0.35, 'width': 0.16, 'height': 0.3}"
},
"graph_understanding": {
"score": 0.74,
"success": false,
"price": 0.01079,
"pass_fail": "Fail",
"response_time": 5.520579099655151,
"result": "```json\n{\n \"A\": {\n \"quantity\": 15,\n \"price\": 20\n },\n \"B\": {\n \"quantity\": 30,\n \"price\": 25\n },\n \"C\": {\n \"quantity\": 40,\n \"price\": 35\n },\n \"D\": {\n \"quantity\": 45,\n \"price\": 42\n }\n}\n```"
},
"color_recognition": {
"score": 0,
"success": false,
"price": 0.00914,
"pass_fail": "Fail",
"response_time": 1.7878320217132568,
"result": "Failed to produce a valid JSON output: I'm sorry, I am not able to provide RGB color codes for elements in images. If you have any other questions or need assistance with something else, feel free to ask!"
},
"annotation_qa": {
"score": 0.33333333333333337,
"success": false,
"price": 0.015300000000000001,
"pass_fail": "Fail",
"response_time": 2.335364580154419,
"result": "```json\n{\n \"missing\": 1\n}\n```"
},
"measurement": {
"score": 0.8571428571428572,
"success": false,
"price": 0.00877,
"pass_fail": "Fail",
"response_time": 4.2275848388671875,
"result": "```json\n{\n \"length\": 3.0,\n \"width\": 3.0\n}\n```"
}
}

0 comments on commit 460cf23

Please sign in to comment.