Skip to content

Commit

Permalink
Improve Modify API page
Browse files Browse the repository at this point in the history
* Remove orphan cleanup instructions entirely. Just mention it exists.
  More detailed material about this should live in pulpcore.
* Modify Repositories page:
  * Add setup step for easy reproduction
  * Normalize presentation
  * Simplify snippets: less subshell "A=$()"
* Misc and typos

[noissue]
  • Loading branch information
pedro-psb committed Apr 5, 2024
1 parent 5758f6e commit fb69a7a
Show file tree
Hide file tree
Showing 6 changed files with 116 additions and 122 deletions.
2 changes: 1 addition & 1 deletion staging_docs/admin/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ Here you'll find information about RPM-specific admin workflows.

If you just got here, consider following the top [Admin Manual](site:pulp-docs/docs/sections/admin/) links, as it provides the common ground for setting up and configuring your Pulp deployment.

You may also find usefull to have a look at [RPM specific Pulp settings](site:pulp_rpm/docs/admin/reference/settings/).
You may also find useful to have a look at [RPM specific Pulp settings](site:pulp_rpm/docs/admin/reference/settings/).

2 changes: 1 addition & 1 deletion staging_docs/dev/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Welcome to Pulp RPM for developers!

Here you'll find information usefull for the RPM plugin developers.
Here you'll find information useful for the RPM plugin developers.

If you just got here, consider following the top [Developer Manual](site:pulp-docs/docs/sections/dev/) links, as it provides the common ground for developers for contributing to docs, to code and getting basic background on plugin development.

26 changes: 3 additions & 23 deletions staging_docs/user/guides/02-upload.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,30 +176,10 @@ pulp rpm content -t CONTENT_TYPE create --help

You may consider [opening a ticket in pulp-cli](https://github.com/pulp/pulp-cli/issues/new/choose) requesting support.



## Delete Content

Removing content from Pulp entirely is slightly complicated for a number of architectural reasons.

1. `repository-versions` are immutable, so you can get back to any version and have all the content available. This means that it is not possible to remove content as long as there still exists a repo-version that "knows about it".
2. Pulp deduplicates content. Artifacts can be shared across repositories - which means the repo being looked at, may not be the only one whose repo-versions still point to the content.

**The way content gets removed "from Pulp" is via the orphan cleanup task**, which will look for Artifacts that don't belong to any repository-versions, and haven't been touched in a while, and removes them.

!!! warning
Please, follow the script steps carefully and only if you know what you are doing because you risk losing data.

```bash
#!/usr/bin/env bash
Deleting *Content* is not part of the user-facing API as a CRUD operation.
There are several architectural reasons for that, one of which is that multiple *Repository Versions* may rely on the same *Content* unit, and forcing a deletion may lead to a broken state.

# Remove content from repository
echo "Removing content from ${REPO_NAME}"
pulp rpm repository version destroy --repository $REPO_NAME
If you want to remove *Content*, consider using the [Modify API](site:pulp_rpm/docs/user/guides/03-modify/#remove-content-from-a-repository) to remove the content *from a Repository*, and not from Pulp itself. The latter should be handled by Admins, who should configure Pulp to handle cleanups safely.

echo "Removing orphan rpm contents"
pulp orphan cleanup --content-hrefs $(pulp rpm content list | jq -cr '.|map(.pulp_href)') --protection-time 0

echo "Checking rpm contents"
pulp rpm repository content list --repository $REPO_NAME
```
196 changes: 107 additions & 89 deletions staging_docs/user/guides/03-modify.md
Original file line number Diff line number Diff line change
@@ -1,144 +1,162 @@
# Modify Repository Content

Modyfing existing Repositorie's Content lets you filter what content you want in a Repository.
This guide will present some methods for achieving that.
Modyfing existing Repository Content lets you filter what content you want in a Repository.

Keep in mind that none of these operations introduces new Content or deletes a Content from a Pulp instance.
To achieve that, see [Post and Delete Content](site:pulp_rpm/docs/user/guides/02-upload/) or [Create, Sync and Publish a Repository](site:pulp_rpm/docs/user/tutorials/01-create_sync_publish/).
To populate Pulp, see [Post and Delete Content](site:pulp_rpm/docs/user/guides/02-upload/) or [Create, Sync and Publish a Repository](site:pulp_rpm/docs/user/tutorials/01-create_sync_publish/).

## Basic Repository Modification API

Like all Pulp repositories, you can use `pulp rpm repository modify` to:

- Add or remove individual content units from a repository by HREF.
- Clone a repository version using `base_version`. This enables roll-back to a previous version.

You'll need to use existing package hrefs or repository versions.
To help you achieve that, you may use `pulp rpm content list`, `pulp rpm repository list` and similar commands.

### Add content to repository
### Sample Setup

If you want to experiment with the operations on some sample data, run this setup so you can follow along.
The output is based on this sample and only makes sense if the operations are followed in order.

```bash
pulp rpm repository create --name modify_test_repo
pulp rpm repository create --name fixture_repo
pulp rpm remote create --name fixture_remote --url https://fixtures.pulpproject.org/rpm-unsigned/
pulp rpm repository sync --repository fixture_repo --remote fixture_remote
```

### Add content to Repository

If there is content already in Pulp, you can add it to a repository using `content modify`.
1. Set required variables:
* `REPONAME`: The repository where you want to add
* `ADD_LIST`: A json list with the `package_href` constructed like:
```json
[{"pulp_href": "/pulp/api/v3/content/rpm/packages/018ea4c6-50f2-7895-aaf9-d1dde2b94c20/"}]
```
2. Run the modify command
3. Inspect the created Repository Version

=== "Add Package to a Repository"
=== "Add Packages to a Repository"

```bash
# Get a Content `pulp_href` and set the href variable
PACKAGE_HREF=CONTENT_PULP_HREF_HERE

# Add created RPM content to repository
echo "Add created RPM Package to repository."
TASK_HREF=$(pulp rpm repository content modify \
--repository "${REPO_NAME}" \
--add-content "[{\"pulp_href\": \"${PACKAGE_HREF}\"}]" \
2>&1 >/dev/null | awk '{print $4}')

# After the task is complete, it gives us a new repository version
echo "Set REPOVERSION_HREF from finished task."
REPOVERSION_HREF=$(pulp show --href "${TASK_HREF}" \
| jq -r '.created_resources | first')
# Set required variables
REPONAME=modify_test_repo
ADD_LIST=$(pulp rpm repository content list \
--repository fixture_repo \
--limit 5 | jq -r '[.[] | {pulp_href}]')

# Run the modify command
pulp rpm repository content modify \
--repository "${REPONAME}" \
--add-content "${ADD_LIST}"

echo "Inspecting RepositoryVersion."
pulp show --href "${REPOVERSION_HREF}"
# Inspect the Repository Version and its Contents
pulp rpm repository show --name modify_test_repo | jq '.latest_version_href'
pulp rpm repository content list --repository modify_test_repo | jq '.[].location_href'
```

=== "Output"

```json
{
"base_version": null,
"content_summary": {
"added": {
"rpm.package": {
"count": 1,
"href": "/pulp/api/v3/content/rpm/packages/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/805de89c-1b1d-432c-993e-3eb9a3fedd22/versions/1/"
}
},
"present": {
"rpm.package": {
"count": 1,
"href": "/pulp/api/v3/content/rpm/packages/?repository_version=/pulp/api/v3/repositories/rpm/rpm/805de89c-1b1d-432c-993e-3eb9a3fedd22/versions/1/"
}
},
"removed": {}
},
"number": 1,
"pulp_created": "2019-11-27T13:48:18.326333Z",
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/805de89c-1b1d-432c-993e-3eb9a3fedd22/versions/1/"
}
# last repository version. Now its 1, previous was 0
"/pulp/api/v3/repositories/rpm/rpm/018ea4da-702b-7b20-b427-393efe196193/versions/1/"

# last repository version content
"zebra-0.1-2.noarch.rpm"
"wolf-9.4-2.noarch.rpm"
"whale-0.2-1.noarch.rpm"
"walrus-5.21-1.noarch.rpm"
"walrus-0.71-1.noarch.rpm"
```

??? tip

It is recommended to omit the `relative_path` and have Pulp generate a common pool location.
This will be `/repo/Packages/s/squirrel-0.1-1.noarch.rpm` as shown below.
### Remove content from a Repository

When specifying a `relative_path`, make sure to add the exact name of the package
including its name, version, release and arch as in `squirrel-0.1-1.noarch.rpm`.
It is composed of the `name-version-release.arch.rpm`.
Removing a content means creating a new *Repository Version* that won't contain it anymore:

Example:

```bash
relative_path="squirrel-0.1-1.noarch.rpm"
1. Set required variables:
* `REPONAME`: The repository where you want to delete from
* `REMOVE_LIST`: A json list with the `package_href` constructed like:
```json
[{"pulp_href": "/pulp/api/v3/content/rpm/packages/018ea4c6-50f2-7895-aaf9-d1dde2b94c20/"}]
```
2. Run the modify command
3. Inspect the created Repository Version

### Remove content from a repository

Removing a content means creating a new repository version that won't contain it anymore.
Again, keep in mind that this doesn't delete the content from Pulp (see how to [Delete Content](#)).

=== "Remove content"
=== "Remove Package from a Repository"

```bash
# Get a Content `pulp_href` and set the href variable
PACKAGE_HREF=CONTENT_PULP_HREF_HERE
# Set required variables
REPONAME=modify_test_repo
REMOVE_LIST=$(pulp rpm repository content list \
--repository modify_test_repo \
--limit 2 | jq -r '[.[] | {pulp_href}]')

# Remove content units from the repository
# Run the modify command
pulp rpm repository content modify \
--repository "${REPO_NAME}" \
--remove-content "[{\"pulp_href\": \"${PACKAGE_HREF}\"}]"
--repository "${REPONAME}" \
--remove-content "${REMOVE_LIST}"

# Inspect the Repository Version and its Contents
pulp rpm repository show --name modify_test_repo | jq '.latest_version_href'
pulp rpm repository content list --repository modify_test_repo | jq '.[].location_href'
```

=== "Output"

```json
TODO
# last repository version. Now its 2, previous was 1
"/pulp/api/v3/repositories/rpm/rpm/018ea4da-702b-7b20-b427-393efe196193/versions/2/"

# last repository version content
"whale-0.2-1.noarch.rpm"
"walrus-5.21-1.noarch.rpm"
"walrus-0.71-1.noarch.rpm"
```

### Copy content to a new repository
### Copy content from a Repository Version

This operation will create a new *Repository Version* in the current Repository based on a previous version (that belongs to the same Repository).
It will contain the exact same contents as in the `base_version`, regardless of what content was previously present.

This operation will create a new repository version in the current repository which is a copy of the one specified as the "base_version", regardless of what content was previously present in the repository.
This can be combined with adding and removing content units in the same call.

=== "Clone a Repository"
1. Set required variables:
* `REPONAME`: The repository to create a copy and get a `base_version` from.
* `REPOVERSION`: The repository version number to roll-back to.
2. Run the modify command
3. Inspect the created Repository Version

=== "Clone a Repository Version"

```bash
# Get a Repository REPOVERSION and set the base-version var
REPOVERSION=BASE_REPOVERSION_HERE
# Set required variables
REPONAME=modify_test_repo
REPOVERSION=1

# Clone a repository
echo "Clone a repository with a content."
TASK_HREF=$(pulp rpm repository content modify \
--repository "${REPO_NAME}" \
--base-version "${REPOVERSION}" \
2>&1 >/dev/null | awk '{print $4}')

# After the task is complete, it gives us a new repository version
echo "Set REPOVERSION_HREF from finished task."
REPOVERSION_HREF=$(pulp show --href "${TASK_HREF}" | jq -r '.created_resources | first')
export REPOVERSION_HREF

echo "Inspecting RepositoryVersion."
pulp show --href "${REPOVERSION_HREF}"
# Run the modify command
pulp rpm repository content modify \
--repository "${REPONAME}" \
--base-version "${REPOVERSION}"

# Inspect the Repository Version and its Contents
pulp rpm repository show --name modify_test_repo | jq '.latest_version_href'
pulp rpm repository content list --repository modify_test_repo | jq '.[].location_href'
```

=== "Output"

```json
TODO
# last repository version. Now its 3, previous was 2
"/pulp/api/v3/repositories/rpm/rpm/018ea4da-702b-7b20-b427-393efe196193/versions/3/"

# last repository version content
"zebra-0.1-2.noarch.rpm"
"wolf-9.4-2.noarch.rpm"
"whale-0.2-1.noarch.rpm"
"walrus-5.21-1.noarch.rpm"
"walrus-0.71-1.noarch.rpm"
```


## Advanced copy workflow

!!! note
Expand Down
10 changes: 3 additions & 7 deletions staging_docs/user/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,14 @@ We also recommended that you read the [Basic Concepts](site:pulp_rpm/docs/user/l

## Features

- `sync-publish-workflow`:
- Sync-publish workflow:
* Support for RPM Packages, Advisories, Modularity, and Comps
* Support for ULN servers
- `Versioned Repositories` so every operation is a restorable snapshot
- `Download content on-demand` when requested by clients to reduce disk space.
- Versioned Repositories so every operation is a restorable snapshot
- Download content on-demand when requested by clients to reduce disk space.
- Upload local RPM content
- Add, remove, copy, and organize RPM content into various repositories
- De-duplication of all saved content
- Host content either [locally or on S3](https://docs.pulpproject.org/installation/storage.html)
- View distributions served by pulpcore-content in a browser

## Requirements

`pulp_rpm` plugin requires some dependencies such as `libsolv` and `libmodulemd`
which is provided only by RedHat family distributions like Fedora.
2 changes: 1 addition & 1 deletion staging_docs/user/learn/01-manage.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ A tree is architecture-specific and is made up of:

Almost all the content in a tree is described in a metadata file called the treeinfo file (sometimes .treeinfo).

There can be only one Distribution Tree in a repository verison.
There can be only one Distribution Tree in a repository version.

## Comps.xml

Expand Down

0 comments on commit fb69a7a

Please sign in to comment.