Add support for remote Firecracker snapshots #1107
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The target branch of this PR is currently stargz-snapshotter, but will be |
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
3 times, most recently
from
b10ef5f to
8139f78
Compare
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
7 times, most recently
from
43fcf55 to
23fd3bd
Compare
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
from
23fd3bd to
817fc62
Compare
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
13 times, most recently
from
b81fc78 to
def6841
Compare
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
4 times, most recently
from
4762bc0 to
a6db1e9
Compare
|
@leokondrashov I'll temporarily change the target of this PR to avoid triggering CI |
andre-j3sus
changed the base branch from
main
to
remote-firecracker-snapshots-poc
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
4 times, most recently
from
c0977e3 to
6eed888
Compare
andre-j3sus
changed the base branch from
remote-firecracker-snapshots-poc
to
main
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
2 times, most recently
from
9f5a856 to
4887968
Compare
andre-j3sus
changed the base branch from
main
to
remote-firecracker-snapshots-poc
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
from
4887968 to
69d7a38
Compare
|
@moharaka please review it first, then notify me to do my pass |
|
@moharaka @leokondrashov sorry, but don't review it yet. There's a test that is not passing in GitHub actions and I need to debug it |
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
4 times, most recently
from
b14ad5d to
644aa3d
Compare
andre-j3sus
changed the base branch from
remote-firecracker-snapshots-poc
to
main
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
from
644aa3d to
455b375
Compare
- When remote snapshots are enabled, after committing the snapshot, it is uploaded to a MinIO instance. When loading from a snapshot, if it is not available locally, it checks if it is available in MinIO and fetches it. - Remote Firecracker snapshots are currently only supported using the Stargz snapshotter (there are some container corruption issues when using devmapper). Signed-off-by: André Jesus <andre.jesus.pilar@gmail.com>
andre-j3sus
force-pushed
the
remote-firecracker-snapshots
branch
from
455b375 to
bdeafcb
Compare
|
@leokondrashov @moharaka This is finally ready for review. The changes are described in the PR description. Feel free to comment if anything looks off—I'm also open to suggestions for design improvements. As for the tests, I know they could be better, but I really can’t spend more time on this PR since I need to start working on the scheduler optimization. |
When set to false and using remote snapshots, snapshots will not be cached locally. The default value is true. Signed-off-by: André Jesus <andre.jesus.pilar@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
When remote snapshots are enabled, after committing a snapshot, it is uploaded to a MinIO instance. When loading from a snapshot, if it is not available locally, the system checks whether it is available in MinIO and fetches it if found.
Closes #823
Implementation Notes ⚒️
-snapshotsoption type from a boolean to a string, with three valid values:local,remote, ordisabled(default).ObjectStoragewith methods to upload and download objects. CreatedMinioStorageas an implementation of this interface.SnapshotManagerto support uploading and downloading snapshots via theObjectStorageinterface.Remote Firecracker snapshots are currently only supported using the Stargz snapshotter (there are known container corruption issues when using devmapper).
External Dependencies 🍀
github.com/minio/minio-go/v7dependency.Breaking API Changes⚠️