Skip to content

Conversation

@stefannibrasil
Copy link
Contributor

@stefannibrasil stefannibrasil commented Nov 13, 2025

Motivation / Background

Closes #3131

Faker::Twitter has never been updated since it was introduced. Not only has the platform's name changed, but their objects have changed a lot.

After working on the user generator, it made sense to also deprecate the status one. The API migration docs don't mention changing status to tweet, but after inspecting what status returns, I realized the new API calls is tweet.

There were lots of changes for both user and tweet objects. Most notably, the shape of the response, additional fields, and removal of deprecated attributes. Used resources as references:

Additional information

I will include a noticeable note in the CHANGELOG when this gets released.

The documentation has been updated to reflect the output changes. The changes were made to match the API's new structure as much as possible, however, I took the liberty of excluding some empty fields that faker included in the past. X's API does not return null fields in their response anymore ("X is adopting the convention that JSON values with no value (for example, null) are not written to the payload. Post and user attributes are only included if they have a non-null values." from Updated JSON design).

Examples of the responses:

Faker::X.user =>
{
  data: [
    {
      author_id: "5688134805624042468",
      id: "2007502004337242257",
      text: "Qui sint magni vel."
    }
  ],
  includes: {
    users: [
      {
        public_metrics: {
          followers_count: 1000,
          following_count: 77,
          tweet_count: 4642,
          listed_count: 704
        },
        username: "lilian",
        pinned_tweet_id: "1702549793917523469",
        entities: {
          url: {
            urls: [ { url: "https://t.co/0iz5wx1ysh", expanded_url: "http://example.com/stuart", display_url: "example.com/stuart" }]
          },
          description: { hashtags: [{tag: "Adipisci"}] }
        },
        description: "Est est laborum dolores.",
        name: "Kimberli Ullrich Jr.",
        verified: false,
        location: "104.82.135.3",
        id: "5688134805624042468",
        protected: false,
        url: "https://t.co/lqsqf67cx5",
        profile_image_url: "https://robohash.org/990174365255127568.png?size=48x48&set=set1",
        created_at: "2018-07-11T00:00:00+00:00"
      }
    ]
  }
}

Faker::X.tweet

Faker::X.tweet =>
{
  data: [{
    id: "5530076569335337477",
    text: "Omnis facere ullam velit.",
    lang: "ja",
    conversation_id: "5530076569335337477",
    created_at: "2009-02-21T07:00:00.000Z",
    author_id: "2788144046134446176",
    public_metrics: {
      retweet_count: 95,
      reply_count: 3,
      like_count: 10,
      quote_count: 3
    },
    possibly_sensitive: false,
    entities: {
      urls: [{
        start: 0,
        end: 5,
        url: "https://t.co/t6o3lav9z1",
        expanded_url: "http://example.com/errol.upton",
        display_url: "example.com/errol.upton"
      }],
      hashtags: [{
        start: 0,
        end: 5,
        tag: "Odit"
      }]
    }
  }]
}

With additional fields:

```ruby
Faker::X.tweet(include_media: true, include_user: true) =>
{
  data: [{
    id: "5340086698567112794",
    text: "Esse nulla minus qui.",
    lang: "en",
    conversation_id: "5340086698567112794",
    created_at: "2009-07-04T06:00:00.000Z",
    author_id: "5156189524741091965",
    public_metrics: {
      retweet_count: 56,
      reply_count: 2,
      like_count: 23,
      quote_count: 1
    },
    possibly_sensitive: false,
    entities: {
      urls: [{
        start: 0,
        end: 5,
        url: "https://t.co/mqplf9rhpn",
        expanded_url: "http://example.com/mohamed_koelpin",
        display_url: "example.com/mohamed_koelpin"
      }],
      hashtags: [{
        start: 0,
        end: 5,
        tag: "Atque"
      }]
    },
    attachments: {
      media_keys: ["6992225089295851582"]
    }
  }],
  includes: {
    media: [{
      height: 526,
      media_key: "6992225089295851582",
      type: "photo",
      preview_image_url: "https://loremflickr.com/1064/600",
      width: 1571,
      alt_text: "Qui ratione magnam et."
    }],
    users: [{
      public_metrics: {
        followers_count: 467,
        following_count: 3,
        tweet_count: 9006,
        listed_count: 984
      },
      username: "gayle",
      pinned_tweet_id: "2282479924658708548",
      entities: {
        url: {
          urls: [{
            start: 0,
            end: 5,
            url: "https://t.co/69eytnuwwu",
            expanded_url: "http://example.com/werner",
            display_url: "example.com/werner"
          }]
        },
        description: {
          hashtags: [{
            start: 0,
            end: 5,
            tag: "Soluta"
          }]
        }
      },
      description: "Esse harum voluptatem voluptate.",
      name: "Elva Spinka",
      verified: false,
      location: "34.230.131.77",
      id: "2365736908578621112",
      protected: false,
      url: "https://t.co/pyuqky3gdl",
      profile_image_url: "https://robohash.org/2204799175591912732.png?size=48x48&set=set1",
      created_at: "2025-01-30T07:00:00.000Z"
    }]
  }
}

Besides renaming the generator to Faker::X,
the generator is vastly outdated. Twitter, aka X API, has changed a lot (it's now v2),
and most of the attributes being returned in the user have been deprecated:
https://docs.x.com/x-api/enterprise-gnip-2.0/fundamentals/data-dictionary#no-longer-supported-deprecated-attributes

To maintain backwards compatibility, users still using Faker::Twitter
will keep using it but be notified of the upcoming changes.

Initially, the goal was to only update the `user` generator but
then I realized it wouldn't make sense to leave `status` as it is.
@stefannibrasil stefannibrasil force-pushed the sb-deprecate-twitter-generator branch from e846c63 to 68d4aa1 Compare November 13, 2025 22:55
After working on the user generator, it made sense to also deprecate
the `status` one. The API migration docs don't mention changing status to tweet,
but after inspecting what `status` returns, I realized the new API calls is `tweet`.

There were lots of changes for both user and tweet objects. Most notably, the shape of the response,
additional fields, and removal of deprecated attributes. Used resources as references:

- https://docs.x.com/x-api/migrate/data-format-migration#user-object
- https://docs.x.com/x-api/fundamentals/data-dictionary#tweet
- https://docs.x.com/x-api/migrate/data-format-migration#entities-and-expanded-entities-objects
@stefannibrasil stefannibrasil force-pushed the sb-deprecate-twitter-generator branch from 68d4aa1 to f2aa395 Compare November 13, 2025 22:56
@stefannibrasil
Copy link
Contributor Author

The change introduced in #3144 is failing for ruby head. If I add the gem version explicitly back again, tests pass locally. Might be an issue that gets fixed down the road when ruby 4.0 is released?

@stefannibrasil stefannibrasil force-pushed the sb-deprecate-twitter-generator branch from cf19b87 to 9b52e23 Compare November 22, 2025 19:17
Copy link
Contributor

@thdaraujo thdaraujo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work! 🙏

@thdaraujo thdaraujo changed the title Update Faker::Twitter: rename to Faker::X and update deprecated attributes Deprecate Faker::Twitter in favor of Faker::X and update attributes Nov 22, 2025
@thdaraujo thdaraujo merged commit 80253e9 into main Nov 22, 2025
7 of 8 checks passed
@thdaraujo thdaraujo deleted the sb-deprecate-twitter-generator branch November 22, 2025 19:56
@thdaraujo
Copy link
Contributor

thdaraujo commented Nov 22, 2025

created an issue #3147 to fix the problem on CI -> https://github.com/faker-ruby/faker/actions/runs/19600283917/job/56130782276?pr=3138

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Faker::Twitter: rename to Faker::X and update deprecated attributes

3 participants