Shootout: ChatGPT, Claude & Grok opinions
I asked all three to comment on the veracity of a quote I read today. Whom do you trust, eh?
I came across a quote today from Scott Dworkin’s site that I wanted to test.
Here’s the quote:
Dem Sens. Elizabeth Warren, Mark Warner, and Jeanne Shaheen sent a letter to AG Pam Bondi which read: “Suggesting that a foreign government adopt Starlink in exchange for relief on tariffs appears to be a textbook case of corruption.” It’s beyond corrupt and it should be illegal.
I asked each of the three LLM-based bots the following question:
How true is the following statement: "Dem Sens. Elizabeth Warren, Mark Warner, and Jeanne Shaheen sent a letter to AG Pam Bondi which read: “Suggesting that a foreign government adopt Starlink in exchange for relief on tariffs appears to be a textbook case of corruption.” It’s beyond corrupt and it should be illegal."
The first two I tried, ChatGPT and Claude, both effectively said they could find no evidence supporting the quote, primarily because their knowledge bases were limited to the end of 2024. But they also added analysis, based on that end date, suggesting the content is unlikely to be correct.
The third, GROK, searched in real time and came up with a very different conclusion:
The statement is partially true but contains inaccuracies and requires clarification based on available information.
Verification of the Statement:
Did Senators Warren, Warner, and Shaheen Send a Letter to AG Pam Bondi?
Yes, U.S. Senators Elizabeth Warren (D-MA), Mark Warner (D-VA), and Jeanne Shaheen (D-NH), along with five other Democratic senators (Jack Reed, Richard Blumenthal, Jeff Merkley, Martin Heinrich, and Ed Markey), sent a letter dated May 14, 2025, to Attorney General Pam Bondi, Acting Director of the Office of Government Ethics Jamieson Greer, White House Designated Agency Ethics Official Scott Gast, and State Department Acting Inspector General Arne Baker. The letter called for an investigation into potential corruption related to the Trump administration’s alleged use of trade negotiations to benefit Elon Musk’s Starlink.
Did the Letter Include the Quoted Statement?
The letter does contain the sentence: “Suggesting that a foreign government adopt Starlink in exchange for relief on tariffs appears to be a textbook case of corruption.” This part of the statement is accurate and directly quoted from the letter as reported by sources.
However, the second part of the quoted statement in your query—“It’s beyond corrupt and it should be illegal”—does not appear in the letter based on the available sources. This phrase seems to be an addition or paraphrase not present in the original text of the letter. The letter focuses on raising concerns about potential ethics violations and corruption but does not use this exact wording.
Context of the Letter:
The letter was prompted by a Washington Post article claiming that the State Department and U.S. embassies had been pushing nations to clear regulatory hurdles for U.S. satellite companies, often specifically mentioning Starlink. The senators expressed concern that Elon Musk, who has a close relationship with President Trump and serves as a senior adviser, might be leveraging his position for personal financial gain through Starlink deals. They cited specific instances, such as Musk’s presence in Saudi Arabia on the same day Starlink received approval there and new U.S. defense and trade agreements were announced. The senators requested investigations into whether Musk or other administration officials violated ethics laws.
Accuracy of the Statement’s Tone and Implications:
The statement’s tone, especially the added phrase “It’s beyond corrupt and it should be illegal,” exaggerates the letter’s language. The letter itself is formal, calling for investigations into potential ethics violations and corruption without making definitive claims about illegality or using emotionally charged language like “beyond corrupt.” The inclusion of this phrase in the statement you provided suggests an embellishment or editorialization not present in the original document.
Critical Examination:
Establishment Narrative: The sources, primarily mainstream media and official Senate press releases, align on the letter’s existence, its recipients, and its core allegations. There is no significant contradiction among these sources, suggesting the letter’s content is accurately reported.
Potential Bias: The letter comes from Democratic senators, who may have political motivations to scrutinize the Trump administration and Musk’s influence. However, their concerns are grounded in specific events (e.g., the Saudi Arabia deal) and supported by a Washington Post report, which lends some credibility to their allegations.
Missing Evidence: The letter does not provide direct evidence of corruption but raises questions based on circumstantial timing and Musk’s dual roles as a private CEO and Trump adviser. Without the investigation they request, it’s premature to label the actions as definitively corrupt or illegal.
Conclusion:
The statement is partially true. The first part of the quoted text (“Suggesting that a foreign government adopt Starlink in exchange for relief on tariffs appears to be a textbook case of corruption”) is accurate and appears in the letter sent by Senators Warren, Warner, Shaheen, and others to Attorney General Pam Bondi and other officials on May 14, 2025. However, the second part (“It’s beyond corrupt and it should be illegal”) is not in the letter and appears to be an inaccurate addition or exaggeration. The letter’s actual content is more measured, calling for investigations into potential ethics violations rather than making conclusive statements about illegality.
If you’d like, I can provide further details about the letter or its context!
25 web pages
So, in this instance, one LLM-based bot appears far more effective than the other two. But there could be other cases where the best in this example is the worst.
[I would never solely rely on a single LLM-based bot to verify content I read about elsewhere. But I wouldn’t ignore the opportunity to test a “shoot out” like this either.]
What do you think of using GROK, Claude or ChatGPT to seek an initial examination of an assertion you might have read?
When I asked the exact question to ChatGPT )4-mini I got this answer, which is more concise but hits the primary point raised by GROK.
That claim is accurate as to the core: on May 14, 2025, Senators Shaheen, Warren, and Warner (joined by several other Democrats) did send a joint letter to Florida Attorney General Pam Bondi (along with the OGE Director, State Department IG, etc.) calling for an ethics probe into reports that the Trump Administration had been pushing Starlink deals in exchange for tariff relief. In that letter they wrote:
“Suggesting that a foreign government adopt Starlink in exchange for relief on tariffs appears to be a textbook case of corruption.”
Senate Foreign Relations Committee
Satellite Today
The follow-on phrase, “It’s beyond corrupt and it should be illegal,” does not appear in the senators’ official letter—it’s commentary you’re seeing from a social-media repost (e.g. X/Twitter). The formal, factual portion is exactly as quoted above.
Sounds like it just takes too much time and effort to carry out this analysis, and for what? Can you depend on the answer? Why not wait for othe sources and perpsectives that might become more valuable as the situation develops - or fades?
Maybe in time this could become a factor - but it becomes ugly - what about planned bias?