Skip to content

Accept limit and offset in enumerate API#6

Open
atharvasaraf123 wants to merge 3 commits into
GoogleCloudPlatform:mainfrom
atharvasaraf123:limit_and_offset
Open

Accept limit and offset in enumerate API#6
atharvasaraf123 wants to merge 3 commits into
GoogleCloudPlatform:mainfrom
atharvasaraf123:limit_and_offset

Conversation

@atharvasaraf123

Copy link
Copy Markdown
Collaborator

This review fixes the issue - google/go-tpm-tools#693
Changes:

  1. Accept limit and offset as query params in enumerate API
  2. Adds test for the above change

@atharvasaraf123

Copy link
Copy Markdown
Collaborator Author

/gcbrun

@NilanjanDaw NilanjanDaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall comment, the API is already published, so we need to make sure current behavior does not change with the addition f the limit and offset parameters. These should be additional optional parameters, and the API should work as-is without them without change in behavior.

Comment thread workload_service/server.go Outdated
Comment thread workload_service/server.go Outdated
if limit == 0 {
limit = 100 // Default pagination limit
}
keys, _, err := s.keyProtectionService.EnumerateKEMKeys(int(limit), int(req.GetOffset()))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if offset is not provided? The API will fail. Since this is a breaking API change we need to make it backward compatible.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetOffset returns 0 if offset is not provided.

if err != nil {
return fmt.Errorf("failed to read request body: %w", err)
}
if len(body) == 0 {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this check for?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do a get for /v1/keys protojson.Unmarshal fails as body is empty. Initially this would work but now that we have limit and offset we need to add this check if we want same behaviour.

Comment thread workload_service/server.go Outdated
}
limit := req.GetLimit()
if limit == 0 {
limit = 100 // Default pagination limit

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, this will change current API behavior limiting to 100 entries if no param is provided.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should the solution be here then? Set this limit to int_max?

Comment thread workload_service/proto/api.proto Outdated
Comment thread workload_service/server.go Outdated
}
limit := req.GetLimit()
if limit == 0 {
limit = 100 // Default pagination limit

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we define a constant defaultEnumerateLimit for this?

Comment thread workload_service/server.go
writeError(w, "offset exceeds maximum allowed value", http.StatusBadRequest)
return
}
keys, _, err := s.keyProtectionService.EnumerateKEMKeys(r.Context(), int32(limit), int32(offset))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're discarding hasMore, the customer won't know if they're on the last page unless they look at the size of the response or call the next page and check if it has 0 entires. we should we sending hasMore in response.

But this would add to the currently shared signature. @NilanjanDaw your thoughts?


message EnumerateKeysRequest {
// The maximum number of key infos to return.
uint64 limit = 1 [(buf.validate.field).uint64 = {gt: 0}];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in workload_service/server.go we have logic to set defaultEnumerateLimit if limit isn't passed.

by default if limit isn't passed, it is considered 0, and it would fail due to this buf validation. we can remove this.

Comment on lines +683 to +685
if limit > math.MaxInt32 {
writeError(w, "limit exceeds maximum allowed value", http.StatusBadRequest)
return

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is dead code now

// The maximum number of key infos to return.
uint64 limit = 1 [(buf.validate.field).uint64 = {gt: 0}];
// The offset to start from.
uint64 offset = 2 [(buf.validate.field).uint64 = {gte: 0}];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uint64 is already gte 0 and won't allow invalid values, so not sure if buf validation is useful here.

@@ -325,6 +325,10 @@ type Server struct {
}

var (

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not related to the PR but should be const

}
}

func TestHandleEnumerateKeysPagination(t *testing.T) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to test:

  • Empty body → should use default limit (100)
  • {"limit": 2000} → should be capped at 1000
  • Malformed JSON body → should return 400

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants