Skip to content

Conversation

ArneBinder
Copy link
Owner

This implements AnnotationList.integrate_predictions(overwrite: bool = False) which allows to move predictions to the main annotations.

@ArneBinder ArneBinder requested a review from ChristophAlt May 7, 2022 19:01
@ChristophAlt
Copy link
Collaborator

Sorry, this is clearly not how this is supposed to be handled. Annotations are groundtruth and predictions are predictions. There should be no need to merge one into the other.

@ArneBinder
Copy link
Owner Author

OK, I can also implement that in my project in a specific serializer.
The background was that Brat does not now the concept of predictions, it has just annotations, and with that PR I could just use my previous serialization scripts without much modification. But it is not so much important, sorry for the noise.

@ArneBinder ArneBinder closed this May 9, 2022
@ArneBinder ArneBinder added the wontfix This will not be worked on label May 9, 2022
@ChristophAlt
Copy link
Collaborator

ChristophAlt commented May 9, 2022

The problem isn't so much to use the predictions as annotations --which we need for multi-step pipelines anyway-- but to deliberately overwrite annotations with predictions. This process should be nondestructive, because there may be actual annotations which then get deleted. Instead one could have a context manager that temporarily maps predictions to annotations without actually deleting or overwriting anything.

with document.predictions_as_annotations(["entities", ...]):
  # do something with document (where document.X now points to document.X.predictions)

or

with document.entities.predictions_as_annotations():
  # do something with document (where document.entities now points to document.entities.predictions)

@ChristophAlt ChristophAlt reopened this May 9, 2022
@ArneBinder ArneBinder changed the title Integrate predictions [WIP] Integrate predictions May 9, 2022
@ArneBinder
Copy link
Owner Author

OK, this sounds really good! Unfortunately, I don't have experience in writing context managers. So, if you want, you can take this over.

@ArneBinder ArneBinder removed the wontfix This will not be worked on label May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants