Merge after delta update

[dmpetrov]

Description

Now we are getting an exception: Delta update cannot be used with merge. It prevents race cond issues. However, we need to enable it in cases when user accepts the risk or runs custom check before merge().

• Introduce reset_delta(), So, chain.reset_delta().merge(...) just works.
• Extend error message like Cannot merge two delta datasets - may cause inconsistency. Use reset_delta() to disable delta tracking.
• Allow merges between delta dataset and not-delta datasets - it cannot lead to race condition
• Make sure all affected commands follow this rules: merge, join, union(?), distinct, etc. (please list it in the docs)

Open question: should chain.clone() reset delta and why?

[ilongin]

Should we maybe call it delta_unsafe() instead of reset_delta() ?
And should we put it just as an argument with other delta arguments?

[dmpetrov]

Should we maybe call it delta_unsafe()

Good idea! It clearly highlights taking risks.

[ilongin]

Should we maybe call it delta_unsafe()

Good idea! It clearly highlights taking risks.

What about having delta_unsafe just as additional argument with others? To me it seems strange that this one is separate as special DataChain method ...

[dmpetrov]

@ilongin I was thinking it's more granular and we have too many params for delta already. And checkpoints are coming with it's own params 🙀

But you are right, for now it might be better to follow the existing pattern and introduce unsafe as a param. Later we can figure out how to extract all the delta params to a separate function.

Let's make it a parameter, not a method.

[shcheklein]

Does delta usafe cover everything here?

[dmpetrov]

We need to check item my item once unsafe is implemented