PipelineModel used in AWS Pipelines

[chubzor]

is it possible to use PipelineModel from sagmaker sdk inside on sagemaker Pipeline steps?

I'm making a pipeline that builds first two and then a third model and in one case I need to use PipelineModel to make a transformer that goes over data. In the other case, I need to combine all 3 models in a PipelineModel which gets deployed to an endpoint.

I'm really not sure how to do that with AWS Pipeline Steps since PipelineModel doesn't really have a signature that can work with CreateModelStep or other AWS Pipelines classes.

[sreedes]

We are adding a list of models as an input to RegisterModelStep, which in turn would construct a PipelineModel internally. Would that work for your case? Alternatively, this aws example provides a way to construct and deploy pipeline models.

[bkoseoglu]

Hi @sreedes, are you currently working on this issue? I have recently had a very similar problem where I need to deploy two models (feature transformation followed by inference) to one endpoint using AWS Pipeline Steps. I couldn't find a way or any sample/instructions to do it using AWS Pipeline Steps

[sreedes]

Yes, here is the PR.

[chubzor]

@sreedes
I'm not sure if docs are updated but maybe you can answer this here:
If I'm using PipelineModel construct for the RegisterModel step, what estimator should I use?
From the two models in list one is a custom script scikit learn and the other is an LinearLearner so I'm not sure what mode_data and estimator should be defined there.
Another question is what if I just want to CreateModel instead of RegisterModel? Is support for CreateModel in the works?

[sreedes]

The estimator could be None, it is now an optional parameter. Earlier Estimator was used to derive model artifacts. Now that the PipelineModel is constructed using Model objects, it is not needed anymore. While creating the Model object, the pipeline properties need to be accordingly set (ie TrainingOutput or ProcessingOutput)

[chubzor]

@sreedes Okay so couple of things to clarify ...
Is there going to be Pipeline support for CreateModel step?

Another question is ... we've got a pipeline that needs to:

1. Train model that generates feature
2. Train LL that parses features
3. Transform our dataset using pipeline of steps 1 and 2
4. Train another categorising model trained with dataset produced by step 3
5. Combine all 3 model into a pipelinemodel and register it as end result.

So how can I get a transformer from the RegisterModel step?
I can't really find it in docs and I don't see a response signature like we have with some other steps?

[Blizter]

@chubzor I found this inside AWS documentation :
https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html#step-type-register-model

Hope it helps