A Spring Cloud Starter for AWS Bedrock

Spring-cloud-aws-bedrock

Simplifying access to Managed Large Language Model Integrations

Introduction

Right on the heels of my release of the Pinecone Springboot Starter earlier last week, I want to talk about an early release of a Spring Cloud Starter for AWS Bedrock.

I want to repeat the importance of Spring as the go-to framework for building robust and scalable Java applications in the enterprise world. Continuing the recognition of the need for a developer-experience-focused spring-boot starter to ease developers into using cutting-edge Generative AI services like Bedrock, I am releasing this starter package/project.

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, along with a broad set of capabilities you need to build generative AI applications, simplifying development while maintaining privacy and security.

Introducing spring-cloud-aws-bedrock-starter, an easy way for Enterprise Spring developers to integrate existing and new Enterprise Applications to AWS Bedrock.

Key Features of the Spring Cloud AWS Bedrock Starter

1. Streamlined Integration: Leveraging the Spring Cloud AWS Starter, the Bedrock Starter eliminates the need for complex boilerplate code, simplifying accessing credentials providers for AWS Bedrock services.
2. Supports various credential models for seamless integration.
3. Activates and provides the Raw BedrockClient and BedrockRuntimeClient preconfigured and ready-to-use.
4. Flexible Service Configuration: For instance, configuring the following properties activates the ClaudeService as a service with methods that can be invoked on the AWS Bedrock

aws.bedrock.model.id=anthropic.claude-v2
aws.bedrock.model.claude.prompt=defaultPrompt
aws.bedrock.model.claude.temperature=0.5
aws.bedrock.model.claude.topP=0.5
aws.bedrock.model.claude.topK=50
aws.bedrock.model.claude.maxTokensToSample=100

Usage and implementation

Ensure to log in to the AWS console, head to the Bedrock home, and request access to Bedrock for the relevant Regions. Dont forget to accept the EULAs before trying to use it.

Incorporating the Spring Cloud AWS Bedrock Starter into your application is straightforward. Services can be auto-wired for easy use. For example, if you have set the Java Application Properties as above, the Autoconfigured ClaudeService is activated and can be injected as below; invoking the Claude model is as simple as well.

private final ClaudeService claudeService;

@Autowired
public ClaudeController(ClaudeService claudeService) {
    this.claudeService = claudeService;
}

@GetMapping("/invoke")
public String invokeClaude(@RequestParam String prompt) {
    try {
        return claudeService.invokeClaude(prompt);
    } catch (Exception e) {
        return "Error invoking Claude: " + e.getMessage();
    }
}

This example showcases the ease of using the Starter to interact with specific Language Models.

The Starter also activates Raw Bedrock and BedrockRuntimeClient objects based on the properties. These can be injected as auto-wired instances and used per the AWS SDK docs for Bedrock as a service

Upcoming releases focus:

1. Support all the models in the bedrock catalog
2. Improving testability
3. Integrating the Starter into the SpringAI project

Conclusion

The Spring Cloud AWS Bedrock Starter is more than a tool; it represents a significant stride in simplifying complex integrations in the enterprise application landscape. As the developer behind this project, I am excited to offer a solution that not only streamlines AI and cloud computing integrations but also paves the way for innovative application development in these fields.

Please be on the lookout for updates on this and other related projects.

Acknowledgements:

• James Williams for getting me excited about Gen AI and kickstarting some of the thought processes here
• Chris Phillipson, with some quick PRs for some code improvements.

References

• spring-cloud-aws-bedrock-starter GitHub Repository
• AWS Bedrock
• Spring Cloud AWS