main_host collector: filter hosts by since & until (created OR modified OR last_automation) #209
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
himdel
added
post-freeze
himdel
force-pushed
the
main_host_since_until
branch
from
ed23771 to
08e252f
Compare
…d OR modified OR last_automation) the idea is to reduce the amount of hosts we collect from all the hosts ever to hosts active during the collection period, but also to allow collecting inactive hosts too so, the `main_host` optional collector remains as-is, using `limit_slicing`, but there's a new optional `main_host_daily` collector, which uses `daily_slicing` That way, you can run a daily cronjob with: ``` METRICS_UTILITY_OPTIONAL_COLLECTORS=main_host_daily ``` and a weekly cronjob with: ``` METRICS_UTILITY_OPTIONAL_COLLECTORS=main_host METRICS_UTILITY_DISABLE_JOB_HOST_SUMMARY_COLLECTOR=true ```
himdel
force-pushed
the
main_host_since_until
branch
from
08e252f to
b76efee
Compare
|
| OR | ||
| (main_host.modified >= '{since.isoformat()}' AND main_host.modified < '{until.isoformat()}') | ||
| OR | ||
| (main_unifiedjob.created >= '{since.isoformat()}' AND main_unifiedjob.created < '{until.isoformat()}') |
There was a problem hiding this comment.
Should we add another OR with main_unifiedjob.finished so we got also all hosts that were used in jobs? Because we are collecting jobs data at the finish of the job, not at the job created.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The idea is to reduce the amount of hosts we collect from all the hosts, ever, to only hosts active during the collection period.
draft: fix typing issues re timezone vs timezoneless sql datetimes
& But we still need inactive host data, just not necessarily daily.
So, TODO: parametrize enabling this,
so people can run daily with since/until enabled, and then separate weekly cronjob for just all-hosts snaphot
& update generator to allow both snapshots and dailies for main_host
the idea is to reduce the amount of hosts we collect from all the hosts ever to hosts active during the collection period,
but also to allow collecting inactive hosts too
so, the
main_hostoptional collector remains as-is, usinglimit_slicing,but there's a new optional
main_host_dailycollector, which usesdaily_slicingThat way, you can run a daily cronjob with:
and a weekly cronjob with:
Fix undefined table handling in main_indirectmanagednodeaudit collector