Skip to content

Commit f65a818

Browse files
kks32kks32
committed
Phase 1+2: reorder nav, consolidate TMS/DB auth, trim duplicates, add missing docs, fix broken APIs
1 parent 6d6f83b commit f65a818

13 files changed

Lines changed: 124 additions & 389 deletions

File tree

‎docs/authentication.md‎

Lines changed: 5 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ dapi authenticates with DesignSafe via the TAPIS v3 API. Credentials are resolve
77
3. `.env` file in your project directory
88
4. Interactive prompts
99

10+
On [DesignSafe JupyterHub](https://jupyter.designsafe-ci.org), authentication is handled automatically. The sections below are for running dapi from a laptop, CI pipeline, or other environment.
11+
1012
## Environment Variables
1113

1214
```bash
@@ -65,87 +67,13 @@ ds = DSClient(
6567
)
6668
```
6769

68-
## TMS Credentials (Execution System Access)
69-
70-
After authenticating with DesignSafe, you need TMS credentials on execution systems where you plan to submit jobs. TMS manages SSH key pairs that allow Tapis to access TACC systems (Frontera, Stampede3, Lonestar6) on your behalf.
71-
72-
:::{note} One-time setup
73-
TMS credentials only need to be established once per system. After that, they persist until you revoke them.
74-
:::
70+
## TMS Credentials
7571

76-
### Establish Credentials
77-
78-
```python
79-
ds = DSClient()
80-
81-
ds.systems.establish_credentials("frontera")
82-
ds.systems.establish_credentials("stampede3")
83-
ds.systems.establish_credentials("ls6")
84-
```
85-
86-
If credentials already exist, `establish_credentials` does nothing (idempotent). To force re-creation:
87-
88-
```python
89-
ds.systems.establish_credentials("frontera", force=True)
90-
```
91-
92-
### Check Credentials
93-
94-
```python
95-
if ds.systems.check_credentials("frontera"):
96-
print("Ready to submit jobs on Frontera")
97-
else:
98-
ds.systems.establish_credentials("frontera")
99-
```
100-
101-
### Revoke Credentials
102-
103-
```python
104-
ds.systems.revoke_credentials("frontera")
105-
```
106-
107-
### Using TMS from Outside DesignSafe
108-
109-
TMS credentials work from any environment -- not just DesignSafe JupyterHub. As long as you can authenticate with Tapis (e.g., via `.env` file), you can manage TMS credentials from your laptop, CI/CD pipelines, or any Python script:
110-
111-
```python
112-
from dapi import DSClient
113-
114-
ds = DSClient()
115-
ds.systems.establish_credentials("frontera")
116-
117-
# Now submit jobs as usual
118-
job_request = ds.jobs.generate(...)
119-
job = ds.jobs.submit(job_request)
120-
```
121-
122-
### Troubleshooting TMS
123-
124-
**Non-TMS System:**
125-
```
126-
CredentialError: System 'my-system' uses authentication method 'PASSWORD', not 'TMS_KEYS'.
127-
```
128-
TMS credential management only works for systems configured with `TMS_KEYS` authentication. TACC execution systems (frontera, stampede3, ls6) use TMS_KEYS.
129-
130-
**System Not Found:**
131-
```
132-
CredentialError: System 'nonexistent' not found.
133-
```
134-
Verify the system ID. Common system IDs: `frontera`, `stampede3`, `ls6`.
72+
After authenticating, dapi needs SSH credentials on TACC execution systems to submit jobs. `DSClient()` sets these up automatically on first use. See [Systems](systems.md) for manual TMS credential management.
13573

13674
## Database Connections
13775

138-
Database connections use built-in public read-only credentials by default -- no `.env` setup is required. To override the defaults (e.g., for a private database instance), set environment variables:
139-
140-
```bash
141-
# Optional: override database credentials
142-
NGL_DB_USER=your_user
143-
NGL_DB_PASSWORD=your_password
144-
NGL_DB_HOST=your_host
145-
NGL_DB_PORT=3306
146-
```
147-
148-
The same pattern applies for VP (`VP_DB_*`) and Earthquake Recovery (`EQ_DB_*`) databases.
76+
Database connections use built-in public read-only credentials by default. No setup is required. See [Database Access](database.md) for override options.
14977

15078
## JWT Token Expiration
15179

@@ -161,8 +89,6 @@ Reinitialize your client to refresh tokens:
16189
ds = DSClient()
16290
```
16391

164-
Tapis tokens have a limited lifespan. Long-running notebooks or scripts will hit this after several hours.
165-
16692
## Troubleshooting
16793

16894
**Invalid credentials:**
@@ -182,16 +108,3 @@ Check your internet connection and DesignSafe service status.
182108
echo $DESIGNSAFE_USERNAME
183109
echo $DESIGNSAFE_PASSWORD
184110
```
185-
186-
## Complete Setup Example
187-
188-
```python
189-
# 1. Create .env file (only Tapis credentials required)
190-
with open('.env', 'w') as f:
191-
f.write('DESIGNSAFE_USERNAME=your_username\n')
192-
f.write('DESIGNSAFE_PASSWORD=your_password\n')
193-
194-
# 2. Initialize client (auto-sets up TMS credentials)
195-
from dapi import DSClient
196-
ds = DSClient()
197-
```

‎docs/database.md‎

Lines changed: 6 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -83,78 +83,21 @@ region_sites = ds.db.ngl.read_sql(
8383

8484
## NGL Database (Next Generation Liquefaction)
8585

86-
### Exploring Tables
87-
8886
```python
89-
tables_info = ds.db.ngl.read_sql("SHOW TABLES")
90-
print(tables_info)
91-
92-
site_structure = ds.db.ngl.read_sql("DESCRIBE SITE")
93-
print(site_structure)
94-
```
87+
# Explore tables
88+
ds.db.ngl.read_sql("SHOW TABLES")
89+
ds.db.ngl.read_sql("DESCRIBE SITE")
9590

96-
### Example Queries
97-
98-
```python
99-
# Active sites
91+
# Query active sites
10092
sites = ds.db.ngl.read_sql("""
101-
SELECT SITE_ID, SITE_NAME, SITE_LAT, SITE_LON, SITE_GEOL
93+
SELECT SITE_ID, SITE_NAME, SITE_LAT, SITE_LON
10294
FROM SITE
10395
WHERE SITE_STAT = 1
10496
ORDER BY SITE_NAME
10597
""")
106-
107-
# Sites with liquefaction data
108-
liquefaction_sites = ds.db.ngl.read_sql("""
109-
SELECT DISTINCT s.SITE_NAME, s.SITE_LAT, s.SITE_LON
110-
FROM SITE s
111-
JOIN RECORD r ON s.SITE_ID = r.SITE_ID
112-
WHERE r.RECORD_STAT = 1
113-
ORDER BY s.SITE_NAME
114-
""")
115-
116-
# Recent earthquakes
117-
earthquakes = ds.db.ngl.read_sql("""
118-
SELECT DISTINCT EVENT_NAME, EVENT_DATE, EVENT_MAG
119-
FROM EVENT
120-
WHERE EVENT_STAT = 1
121-
ORDER BY EVENT_DATE DESC
122-
LIMIT 20
123-
""")
124-
125-
# CPT data summary
126-
cpt_summary = ds.db.ngl.read_sql("""
127-
SELECT
128-
COUNT(*) as total_cpts,
129-
AVG(CPT_DEPTH) as avg_depth,
130-
MIN(CPT_DEPTH) as min_depth,
131-
MAX(CPT_DEPTH) as max_depth
132-
FROM CPT
133-
WHERE CPT_STAT = 1
134-
""")
13598
```
13699

137-
### Joins
138-
139-
```python
140-
# Sites with multiple liquefaction events
141-
high_risk_sites = ds.db.ngl.read_sql("""
142-
SELECT
143-
s.SITE_NAME,
144-
s.SITE_LAT,
145-
s.SITE_LON,
146-
COUNT(l.LIQ_ID) as liquefaction_events,
147-
AVG(e.EVENT_MAG) as avg_magnitude
148-
FROM SITE s
149-
JOIN RECORD r ON s.SITE_ID = r.SITE_ID
150-
JOIN LIQUEFACTION l ON r.RECORD_ID = l.RECORD_ID
151-
JOIN EVENT e ON r.EVENT_ID = e.EVENT_ID
152-
WHERE s.SITE_STAT = 1 AND r.RECORD_STAT = 1
153-
GROUP BY s.SITE_ID
154-
HAVING liquefaction_events > 2
155-
ORDER BY liquefaction_events DESC, avg_magnitude DESC
156-
""")
157-
```
100+
For more NGL queries (joins, geographic filtering, liquefaction analysis), see the [Database example](examples/database.md).
158101

159102
## Earthquake Recovery Database
160103

‎docs/examples.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Examples
22

3+
Complete worked examples showing dapi workflows. Each links to a runnable notebook on DesignSafe JupyterHub.
4+
35
### Application Management
46
Discover and manage applications on DesignSafe.
57

0 commit comments

Comments
 (0)