End to end tests

[kvanbere]

• Build an example that automatically scrapes JSON payloads for each type of event (perhaps use the GitHub package to trigger the events?). This will be useful to keep up to date with undocumented changes to the GitHub API
• Implement ToJSON instances for each of the payloads
• Run end to end tests that verify the result of running ‘ToJSON . FromJSON’ is the same as what gets scraped. If not, fields are missing

[Exegetech]

@donkeybonks Seems that the second point is the easiest. Could you explain more?

[kvanbere]

Yes, we will need to decouple these from the smoke tests. So it will be a new file under spec/.

But the general idea is that we could implement ToJSON instead of just FromJSON and then have suites of tests that test toJSON . fromJSON == id for each type.

This means that if you have a field that is not handled on the input fixture (see fixtures/), it will not be decoded and thus when it is re-encoded it will be missing so toJSON . fromJSON /= id and we know something is wrong.

The first part is simply automating the generation of the fixtures/ folder. Those fixtures are pulled from the GitHub documentation but it seems to be somewhat out of date.. missing a few fields added in the last year or two.

It is reasonable to decouple the two and implement them as separate issues.

[Exegetech]

Ah that makes sense. Basically some sort of property basic testing.

I see, it looks that it is better to do the first part first. How do we pull that currently? Is it done manually?

[kvanbere]

It's pulled from here: https://developer.github.com/webhooks/

It is reasonable to do the second part first. The first part is non-trivial unfourtunately

[Exegetech]

Sounds good, I'll try the second part.

[Exegetech]

@donkeybonks I looked at spec/DecodeEventsSpec.hs and curious about the lines on top of the file.

I tried to run,

-- Run GHCi:
-- $ stack install pretty-show
-- $ stack ghci --test pretty-show

Then I run these imports

-- Paste:
{-
import Data.Aeson
import qualified Data.ByteString.Lazy as BSL
import GitHub.Data.Webhooks.Events
import Text.Show.Pretty
import Text.PrettyPrint.HughesPJ
-}

This is what I get

Warning: /Users/christiansakai/Desktop/random/github-webhooks/github-webhooks.cabal was
         generated with a newer version of hpack, please upgrade and try again.

Warning: /Users/christiansakai/Desktop/random/github-webhooks/examples/servant-simple/github-webhooks-servant-simple-example.cabal
         was generated with a newer version of hpack, please upgrade and try again.

Warning: /Users/christiansakai/Desktop/random/github-webhooks/examples/servant/github-webhooks-servant-example.cabal
         was generated with a newer version of hpack, please upgrade and try again.

Warning: /Users/christiansakai/Desktop/random/github-webhooks/examples/scotty/github-webhooks-scotty-example.cabal
         was generated with a newer version of hpack, please upgrade and try again.

Warning: Some targets (pretty-show) are not local packages, and so cannot be directly loaded.
         In future versions of stack, this might be supported - see
         https://github.com/commercialhaskell/stack/issues/1441 . It can still be useful to
         specify these, as they will be passed to ghci via -package flags.

Warning: No local targets specified, so ghci will not use any options from your package.yaml
         / *.cabal files.

         Potential ways to resolve this:
         * If you want to use the package.yaml / *.cabal package in the current directory,
           use stack init to create a new stack.yaml.
         * Add to the 'packages' field
           of /Users/christiansakai/Desktop/random/github-webhooks/stack.yaml

Configuring GHCi with the following packages:
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /Users/christiansakai/dotfiles/.ghci
Loaded GHCi configuration from /private/var/folders/nr/rmw0xhw10s3fndfk237g0mdc0000gn/T/ghci64069/ghci-script
λ: import Data.Aeson

<no location info>: error:
    Could not find module ‘Data.Aeson’
    Perhaps you meant Data.Version (from base-4.10.1.0)
λ: import qualified Data.ByteString.Lazy as BSL

<no location info>: error:
    Could not find module ‘Data.ByteString.Lazy’
    It is not a module in the current program, or in any known package.
λ: import Github.Data.Webhooks.Events

<no location info>: error:
    Could not find module ‘Github.Data.Webhooks.Events’
    It is not a module in the current program, or in any known package.
λ: import Text.Show.Pretty
λ: import Text.PrettyPrint.HughesPJ

<no location info>: error:
    Could not find module ‘Text.PrettyPrint.HughesPJ’
    It is not a module in the current program, or in any known package.
λ:

However, if I just do stack ghci

The I tried to import

Configuring GHCi with the following packages: github-webhooks
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /Users/christiansakai/dotfiles/.ghci
[1 of 3] Compiling GitHub.Data.Webhooks.Payload ( /Users/christiansakai/Desktop/random/github-webhooks/src/GitHub/Data/Webhooks/Payload.hs, interpreted )
[2 of 3] Compiling GitHub.Data.Webhooks.Events ( /Users/christiansakai/Desktop/random/github-webhooks/src/GitHub/Data/Webhooks/Events.hs, interpreted )
[3 of 3] Compiling GitHub.Data.Webhooks.Secure ( /Users/christiansakai/Desktop/random/github-webhooks/src/GitHub/Data/Webhooks/Secure.hs, interpreted )
Ok, three modules loaded.
Loaded GHCi configuration from /private/var/folders/nr/rmw0xhw10s3fndfk237g0mdc0000gn/T/ghci64979/ghci-script
λ: import Data.Aeson
λ: import qualified Data.ByteString.Lazy as BSL
λ: import GitHub.Data.Webhooks.Events
λ: import Text.Show.Pretty

<no location info>: error:
    Could not find module ‘Text.Show.Pretty’
    It is not a module in the current program, or in any known package.
λ: import Text.PrettyPrint.HughesPJ

<no location info>: error:
    Could not find module ‘Text.PrettyPrint.HughesPJ’
    It is not a module in the current program, or in any known package.
λ:

So:

• For stack ghci, basically if I understand this correctly. stack ghci builds the current package where the command was invoked in, in this case github-webhooks and its dependencies, in this case, aeson and bytestring are dependencies of github-webhooks, that's why I was able to import it.

  However, Text.Show.Pretty (pretty-show package) and Text.PrettyPrint.HughesPJ (pretty package) are not the dependencies of github-webhooks.

• For stack ghci --test pretty-show, I don't quite understand why it doesn't work. Could you help me?

[kvanbere]

Hmm, the behavior of stack has changed here .. or perhaps it is with the latest resolver (for ghci) that it has changed.

You can manually specify all the packages required as a work around, even though the prompt says you can't. ie

$ stack --test ghci pretty pretty-show github-webhooks aeson bytestring

YMMV:

kvanb at donkeybonks in ~/git/github-webhooks on master
$ stack --test ghci pretty pretty-show github-webhooks aeson bytestring

Warning: /Users/kvanb/git/github-webhooks/github-webhooks.cabal was generated with a newer version of hpack,
         please upgrade and try again.

Warning: /Users/kvanb/git/github-webhooks/examples/servant-simple/github-webhooks-servant-simple-example.cabal
         was generated with a newer version of hpack, please upgrade and try again.

Warning: /Users/kvanb/git/github-webhooks/examples/servant/github-webhooks-servant-example.cabal was generated
         with a newer version of hpack, please upgrade and try again.

Warning: /Users/kvanb/git/github-webhooks/examples/scotty/github-webhooks-scotty-example.cabal was generated with
         a newer version of hpack, please upgrade and try again.
github-webhooks-0.9.1: initial-build-steps (lib + test)
github-webhooks-0.9.1: Test running disabled by --no-run-tests flag.
Completed 2 action(s).

Warning: Some targets (aeson, bytestring, pretty, pretty-show) are not local packages, and so cannot be directly
         loaded. In future versions of stack, this might be supported - see
         https://github.com/commercialhaskell/stack/issues/1441 . It can still be useful to specify these, as
         they will be passed to ghci via -package flags.
Configuring GHCi with the following packages: github-webhooks
Using main module: 1. Package `github-webhooks' component test:spec with main-is file: /Users/kvanb/git/github-webhooks/spec/Spec.hs
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
[1 of 5] Compiling GitHub.Data.Webhooks.Payload ( /Users/kvanb/git/github-webhooks/src/GitHub/Data/Webhooks/Payload.hs, interpreted )
[2 of 5] Compiling GitHub.Data.Webhooks.Events ( /Users/kvanb/git/github-webhooks/src/GitHub/Data/Webhooks/Events.hs, interpreted )
[3 of 5] Compiling DecodeEventsSpec ( /Users/kvanb/git/github-webhooks/spec/DecodeEventsSpec.hs, interpreted )
[4 of 5] Compiling GitHub.Data.Webhooks.Secure ( /Users/kvanb/git/github-webhooks/src/GitHub/Data/Webhooks/Secure.hs, interpreted )
[5 of 5] Compiling Main             ( /Users/kvanb/git/github-webhooks/spec/Spec.hs, interpreted )
Ok, five modules loaded.
Loaded GHCi configuration from /private/var/folders/bj/fmlb9nv10yvf3h7m2vsh_5m80000gn/T/ghci2533/ghci-script
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure> import Data.Aeson
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure Data.Aeson> import qualified Data.ByteString.Lazy as BSL
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure Data.Aeson BSL> import GitHub.Data.Webhooks.Events
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure Data.Aeson BSL> import Text.Show.Pretty
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure Data.Aeson BSL Text.Show.Pretty> import Text.PrettyPrint.HughesPJ
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload 

GitHub.Data.Webhooks.Secure Data.Aeson BSL Text.Show.Pretty Text.PrettyPrint.HughesPJ> do { x <- BSL.readFile "fixtures/watch-event.json"; putStrLn $ renderStyle (style {lineLength=200}) . ppDoc $ either error id (eitherDecode' x :: Either String WatchEvent) }
WatchEvent
  { evWatchAction = WatchStartedAction
  , evWatchRepo =
      HookRepository
        { whRepoId = 35129377
        , whRepoName = "public-repo"
        , whRepoFullName = "baxterthehacker/public-repo"
        , whRepoOwner =
            Right
              HookUser
                { whUserLogin = "baxterthehacker"
                , whUserId = 6752317
                , whUserAvatarUrl = URL "https://avatars.githubusercontent.com/u/6752317?v=3"
                , whUserGravatarId = URL ""
                , whUserUrl = URL "https://api.github.com/users/baxterthehacker"
                , whUserHtmlUrl = URL "https://github.com/baxterthehacker"
                , whUserFollowersUrl = URL "https://api.github.com/users/baxterthehacker/followers"
                , whUserFollowingUrl = URL "https://api.github.com/users/baxterthehacker/following{/other_user}"
                , whUserGistsUrl = URL "https://api.github.com/users/baxterthehacker/gists{/gist_id}"
                , whUserStarredUrl = URL "https://api.github.com/users/baxterthehacker/starred{/owner}{/repo}"
                , whUserSubscriptionsUrl = URL "https://api.github.com/users/baxterthehacker/subscriptions"
                , whUserOrganizationsUrl = URL "https://api.github.com/users/baxterthehacker/orgs"
                , whUserReposUrl = URL "https://api.github.com/users/baxterthehacker/repos"
                , whUserEventsUrl = URL "https://api.github.com/users/baxterthehacker/events{/privacy}"
                , whUserReceivedEventsUrl = URL "https://api.github.com/users/baxterthehacker/received_events"
                , whUserType = OwnerUser
                , whUserIsAdminOfSite = False
                }
        , whRepoIsPrivate = False
        , whRepoHtmlUrl = URL "https://github.com/baxterthehacker/public-repo"
        , whRepoDescription = ""
        , whRepoIsAFork = False
        , whRepoUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo"
        , whRepoForksUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/forks"
        , whRepoKeysUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/keys{/key_id}"
        , whRepoCollaboratorsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/collaborators{/collaborator}"
        , whRepoTeamsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/teams"
        , whRepoHooksUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/hooks"
        , whRepoIssueEventsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/issues/events{/number}"
        , whRepoEventsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/events"
        , whRepoAssigneesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/assignees{/user}"
        , whRepoBranchesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/branches{/branch}"
        , whRepoTagsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/tags"
        , whRepoBlobsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/git/blobs{/sha}"
        , whRepoGitTagsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/git/tags{/sha}"
        , whRepoGitRefsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/git/refs{/sha}"
        , whRepoTreesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/git/trees{/sha}"
        , whRepoStatusesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/statuses/{sha}"
        , whRepoLanguagesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/languages"
        , whRepoStargazersUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/stargazers"
        , whRepoContributorsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/contributors"
        , whRepoSubscribersUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/subscribers"
        , whRepoSubscriptionUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/subscription"
        , whRepoCommitsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/commits{/sha}"
        , whRepoGitCommitsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/git/commits{/sha}"
        , whRepoCommentsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/comments{/number}"
        , whRepoIssueCommentsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/issues/comments{/number}"
        , whRepoContentsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/contents/{+path}"
        , whRepoCompareUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/compare/{base}...{head}"
        , whRepoMergesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/merges"
        , whRepoArchiveUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/{archive_format}{/ref}"
        , whRepoDownloadsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/downloads"
        , whRepoIssuesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/issues{/number}"
        , whRepoPullsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/pulls{/number}"
        , whRepoMilestonesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/milestones{/number}"
        , whRepoNotificationsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/notifications{?since,all,participating}"
        , whRepoLabelsUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/labels{/name}"
        , whRepoReleasesUrl = URL "https://api.github.com/repos/baxterthehacker/public-repo/releases{/id}"
        , whRepoCreatedAt = 2015 (-05) (-05) 23 : 40 : 12 UTC
        , whRepoUpdatedAt = 2015 (-05) (-05) 23 : 40 : 30 UTC
        , whRepoPushedAt = 2015 (-05) (-05) 23 : 40 : 27 UTC
        , whRepoGitUrl = URL "git://github.com/baxterthehacker/public-repo.git"
        , whRepoSshUrl = URL "git@github.com:baxterthehacker/public-repo.git"
        , whRepoCloneUrl = URL "https://github.com/baxterthehacker/public-repo.git"
        , whRepoSvnUrl = URL "https://github.com/baxterthehacker/public-repo"
        , whRepoHomepage = Nothing
        , whRepoSize = 0
        , whRepoStargazersCount = 0
        , whRepoWatchersCount = 0
        , whRepoLanguage = Nothing
        , whRepoHasIssues = True
        , whRepoHasDownloads = True
        , whRepoHasWiki = True
        , whRepoHasPages = True
        , whRepoForkCount = 0
        , whRepoMirrorUrl = Nothing
        , whRepoOpenIssuesCount = 2
        , whRepoDefaultBranchName = "master"
        }
  , evWatchSender =
      HookUser
        { whUserLogin = "baxterthehacker"
        , whUserId = 6752317
        , whUserAvatarUrl = URL "https://avatars.githubusercontent.com/u/6752317?v=3"
        , whUserGravatarId = URL ""
        , whUserUrl = URL "https://api.github.com/users/baxterthehacker"
        , whUserHtmlUrl = URL "https://github.com/baxterthehacker"
        , whUserFollowersUrl = URL "https://api.github.com/users/baxterthehacker/followers"
        , whUserFollowingUrl = URL "https://api.github.com/users/baxterthehacker/following{/other_user}"
        , whUserGistsUrl = URL "https://api.github.com/users/baxterthehacker/gists{/gist_id}"
        , whUserStarredUrl = URL "https://api.github.com/users/baxterthehacker/starred{/owner}{/repo}"
        , whUserSubscriptionsUrl = URL "https://api.github.com/users/baxterthehacker/subscriptions"
        , whUserOrganizationsUrl = URL "https://api.github.com/users/baxterthehacker/orgs"
        , whUserReposUrl = URL "https://api.github.com/users/baxterthehacker/repos"
        , whUserEventsUrl = URL "https://api.github.com/users/baxterthehacker/events{/privacy}"
        , whUserReceivedEventsUrl = URL "https://api.github.com/users/baxterthehacker/received_events"
        , whUserType = OwnerUser
        , whUserIsAdminOfSite = False
        }
  }
*Main DecodeEventsSpec GitHub.Data.Webhooks.Events GitHub.Data.Webhooks.Payload GitHub.Data.Webhooks.Secure Data.Aeson BSL Text.Show.Pretty Text.PrettyPrint.HughesPJ>