feat: CLI: provide a way to list agents from the command line with filtering

[ajzobro]

Description

Adds list-agents sub-command to the testflinger CLI to allow for listing agents which match specified criteria, and presenting that information either as a single column of the agents' name, a table of agent names and attributes, or a summary of the agent statuses matching the specified filter.

usage: testflinger list-agents [-h] [-1] [--summary] [--filter-status FILTER_STATUS] [--filter-name FILTER_NAME]
                               [--filter-queues FILTER_QUEUES] [--filter-location FILTER_LOCATION]
                               [--filter-provision-type FILTER_PROVISION_TYPE] [--filter-comment FILTER_COMMENT]
                               [--fields FIELDS]

options:
  -h, --help            show this help message and exit
  -1                    Single agent name per line (suitable for piping)
  --summary             Show summary of online/offline agent counts
  --filter-status FILTER_STATUS
                        Filter agents by status (comma-separated). Use ^ prefix to exclude.
                        gross: online,offline,maintenance or 
                        fine: waiting,setup,provision,firmware_update,test,allocate,reserve,cleanup.
                        Example: --filter-status online,^waiting
  --filter-name FILTER_NAME
                        Filter agents by name (regex)
  --filter-queues FILTER_QUEUES
                        Filter agents by queues (regex, matches any queue)
  --filter-location FILTER_LOCATION
                        Filter agents by location (regex)
  --filter-provision-type FILTER_PROVISION_TYPE
                        Filter agents by provision-type (regex)
  --filter-comment FILTER_COMMENT
                        Filter agents by comment (regex)
  --fields FIELDS       Fields to display in the agent table (comma-separated). Available fields: name, status,
                        location, provision_type, comment, job_id, queues. Default:
                        name,status,location,provision_type,comment

There are three main modes of operation:

• Single-column list mode
• Table mode
• Summary mode

Single-column list mode (-1)

In single-column list mode (-1) output is suitable for further processing by shell scripts in very much the same way as ls -1:

for agent in $(uv run testflinger list-agents --filter-queue "petilil|audino" -1); do testflinger admin set agent-status --agents $agent --status online; done
Agent audino status is now: waiting
Agent petilil status is now: waiting

Table mode (default)

If neither -1 nor --summary are specified, the default is to output a table of agents matching the specified filters.

run testflinger list-agents
Name     Status   Location  Provision Type  Comment
---------------------------------------------------
audino   waiting  TXR3-DH1  maas2                  
multi-3  waiting            multi                  
petilil  waiting  TXR3-DH1  maas2                  

Summary mode (--summary)

testflinger list-agents --summary
Online:           2413
  waiting          2328
  provision        28
  test             30
  reserve          27

Offline:          63
  offline          55
  maintenance      8

Resolved issues

Resolves https://warthogs.atlassian.net/browse/CERTTF-884 (#982)
Resolves https://warthogs.atlassian.net/browse/CERTTF-778

Documentation

Updated agents.rst to include the list-agents subcommand.

Web service API changes

N/A

Tests

Unit tests were added. See above for example usage as well.

[codecov]

Codecov Report

❌ Patch coverage is 95.48387% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.00%. Comparing base (cca74db) to head (151a0ae).
⚠️ Report is 14 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #986      +/-   ##
==========================================
+ Coverage   75.31%   76.00%   +0.68%     
==========================================
  Files         110      111       +1     
  Lines       10808    11172     +364     
  Branches      899      929      +30     
==========================================
+ Hits         8140     8491     +351     
- Misses       2474     2483       +9     
- Partials      194      198       +4     

Flag	Coverage Δ		*Carryforward flag
agent	75.78% <ø> (+1.06%)	⬆️
cli	90.94% <95.48%> (+0.53%)	⬆️
device	62.80% <ø> (ø)		Carriedforward from 6c48210
server	88.10% <ø> (ø)		Carriedforward from 6c48210

*This pull request uses carry forward flags. Click here to find out more.

Components	Coverage Δ
Agent	75.78% <ø> (+1.06%)	⬆️
CLI	90.94% <95.48%> (+0.53%)	⬆️
Common	∅ <ø> (∅)
Device Connectors	62.80% <ø> (ø)
Server	88.10% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ajzobro]

Update to use JobState enums introduced into the CLI by #988 These were not added and instead #993 was created.

[Hook25]

got nerd-sniped into giving this a review. Good job here, I have a few inline suggestions here and there but overall lgtm. Only major thing I would change is that I wouldn't put the help output in the howto due to bitrot, it is not very howto-ish and it is not quite clear what purpose it serves.

[Hook25]

this is a nargs with choice and an array default, please. Else the errors are wacky and late + you have to do more inline validation in the logic below. Also, use the default dumper if possible to print defaults in help.

[ajzobro]

Following the spec for CLI, multi-value args should either use commas or repeated args. For these which have specific sets of values the preference is for a comma separated list.

I have moved this to it's own type parser for argparse.

[Hook25]

I think that is not idiomatic in any language as far as I know, so I would rather not follow such a bad guidance to be honest. Your call. I would just use nargs and ignore that.

[ajzobro]

Thanks for the review!

[ajzobro]

Following the spec for CLI, multi-value args should either use commas or repeated args. For these which have specific sets of values the preference is for a comma separated list.

I have moved this to it's own type parser for argparse.

[tang-mm]

Thanks for the addition! The new command is very well explained and organized. I left a couple of comments on the structure, please adjust them as needed

[tang-mm]

In the section "Check Available Queues on the Server" below, the tutorial shows usage of list-queues, but it only returns a few advertised queues. Does it make sense to switch to showing the new list-agents?

[ajzobro]

@rene-oromtz What are your thoughts on swapping these (list-queues) vs (list-agents)?

[rene-oromtz]

Two comments about this:

1. Based on a previous comment from Max, should we keep also the --help here? Would it be best to generalize and say if no error returned you are good to go? Otherwise whenever adding new commands, we need to update this (which is easy to forgot, at least I did for the admin command.

2. As for the Check Available Queues on the Server section I think its a good idea to modify this section to make it actually useful with this new addition! We now have all the tools to find an appropriate queue for testing. We can now:

• Get a list of available agents first list-agents (with filtering per data center, provision type etc)
• Once you identify your agent, you can get the queues the agent is listening with agent-status <agent> --json | jq .queues
• Use any of those queues for your job.

We should eventually make list-queues to actually work, but at least right now we can provide value to users.

[rene-oromtz]

Aaand solve one of the pain points in #661 (No example queue)

[ajzobro]

#1 can be fixed with scripting (actions to keep the --help updated) -- I think the --help is useful (if up-to-date). I removed it from other places because those places were a 'how-to' and not a 'reference' section.

#2, just update --fields to show queues

Can update in that future ticket to output --json can still be used.

[rene-oromtz]

1. In this case, the help is just to indicate which are the options from the cli and to know it was installed successfully. For this purpose, I think we can keep a similar approach as juju docs tutorial to just say something similar as:

...
sudo snap install testflinger-cli 
testflinger-cli 20260205 from Canonical Certification Team (ce-certification-qa) installed <- we can include this to explicitly let the user know the snap was installed successfully
... 
Our testflinger-cli is ready! Take a quick look at what it can do: testflinger-cli --help <- we can add this to let the users know how to run the help in case they want to do something else not covered in the tutorial

that way no need to add another workflow or manage it manually but probably want @tang-mm inputs on this to see what is more maintainable on the long run.

2. Sounds good, at least the general users should be able to use the new command to effectively get the queues. Once --json is included, CI execution can leverage as well of this command output

[Hook25]

remember to purge the commented out code, consider the regex thingy which is pretty minor imo but a nice to have

[rene-oromtz]

I left just a few comments but overall I think the feedback from Max and Meng Meng has been great and will let them give the final approval.

LGTM but In general, I think this could also benefit a lot if we do the filtering server side at the database level instead of client side e.g. GET /v1/agents/data with query parameters instead of getting all agents data and letting the client process it (it might be even faster). Not a hard requirement now because it will involve redesigning this PR and I think eventually we would need to make the API more robust anyway. Just a comment to maybe keep it as a future enhancement.

[rene-oromtz]

Thanks for the new pages, this were really needed! I focused on reviewing the docs changes that came from my earlier comments. I added some comments but I think overall for some of them I would want to see @tang-mm opinion on what is best.

[rene-oromtz]

Since this is already an explanation for agents, I think this is not entirely required imo, consider removing this subsection

[rene-oromtz]

Just checked below that your intention is to clarify the difference between agent vs agent host. Maybe this could use an introduction right below the H1 heading (line 5). Something that indicates the relation between agents and agent host. Then maybe this can just become:

Suggested change

	Agents in general
	-----------------
	Agents
	------

[rene-oromtz]

Based on my previous comment this can maybe be generalized to indicate the relation between agent and agent hosts and uses as introduction to this page

[rene-oromtz]

Based on previous comments, this can then become:

Suggested change

	What is an Agent Host?
	~~~~~~~~~~~~~~~~~~~~~~
	Agent Host
	----------

[rene-oromtz]

This administration sectio is already covered in the new manage-agents.rst isnt? Maybe something pending to be removed? Not sure if something is missing but consider to review and check which things are needed to be removed/move to the new page

[ajzobro]

We are in the agents explanation file. I don't know if it should EXPLAIN everything or if it should just point to HOW-TO documents where these things are demonstrated and also EXPLAINED again.

I will wait for @tang-mm to provide feedback before I make any changes because there are a lot of different ways we could do this and I don't want to draw this out guessing.

[ajzobro]

@rene-oromtz ready for another look