Open source AI chatbot privacy issues addressed by Sendbird

Last month Sendbird became the first communications API platform to integrate Llama2, Meta’s open source LLM. By adding support for Llama2, on top of existing ChatGPT and PaLM-2 integrations, Sendbird became the first platform of its kind to offer organizations greater flexibility for their integration of generative AI chatbots. By doing so, companies can choose the option that best suits their overall strategy and security preferences, giving them more control over communications experiences.

We were particularly interested in Sendbird’s choice to integrate Llama2 and what advantages it could provide so we dove in with the company to learn more.

ADM: What sets Llama2 apart from other commercially available LLMs, and why did Sendbird choose to integrate it?

Kim: Llama2 distinguishes itself not only through its remarkable performance in reasoning, knowledge, and programming, achieved by training on an extensive dataset of 2 trillion tokens and 1 million human annotations, but also through its open-source nature. This transparency provides valuable insights into the LLM’s architecture and training methodologies, allowing developers to directly address compliance and ethical concerns.

As a standalone, downloadable LLM, Llama2 offers the capability to run on-premise or on a private cloud, significantly reducing the risk of data exposure. Additionally, Llama2’s comes in various sizes, with the smallest version capable of running efficiently on a mobile device without requiring an internet connection.

Furthermore, Llama2 can be fine-tuned with proprietary data, making it available for custom needs. Sendbird as the communications infrastructure provider already handles clients’ various kinds of proprietary data like conversations that can be fed directly into the model for fine-tuning. So Sendbird can continuously improve the model for our clients in an end-to-end manner.

Sendbird’s decision to integrate Llama2 was driven by these compelling attributes—its superior performance, enhanced privacy and security features, and remarkable flexibility. More importantly, the integration of Llama2 aligns with our commitment to offering a broad spectrum of alternatives that can cater to the diverse needs of our customers, ensuring that we provide solutions that are not just effective but also aligned with their specific privacy requirements and values.

ADM: The blog mentions that Llama2 is open-source. How does open source LLMs align with Sendbird’s broader product strategy and mission?

Kim: Open-source LLMs like Llama2 align with Sendbird’s commitment to privacy and transparency. By hosting Llama2 in our own cloud, we ensure that conversations are private and not shared with third parties, aligning with our mission to provide secure and reliable communication solutions.

ADM: Sendbird already had an existing LLM offering. What gaps or needs does the Llama2 integration specifically address?

Kim: The integration of Llama2 specifically addresses the need for enhanced privacy and security in our LLM offerings. While our existing solutions also offer fine-tuning capabilities, Llama2 offers a significant advantage by allowing the LLM to run on a private cloud or on-premise. This ensures data security and strict control over the conversational AI system. By preventing data to leave their controlled environment, businesses can eliminate the risk of data exposure and breaches. This level of privacy and security is a key differentiator for Llama2, making it an ideal solution for industries and organizations where data confidentiality and regulatory compliance are paramount.

ADM: Could you elaborate on the customization features that the Llama2 integration will offer to developers?

Kim: The integration of Llama2 into our platform offers developers a highly scalable and customizable conversational AI solution, tailored to meet the distinct needs of various businesses.

Adaptability: Thanks to Llama2’s open-source framework, it provides unparalleled adaptability. Developers can make industry-specific modifications or fine-tune the model for particular use cases. This flexibility ensures that the resulting chatbot is perfectly aligned with a company’s business objectives and operational nuances.

User Experience Control: A pivotal aspect of our bot integration is the harmonious blend of features that harness the bot’s creativity while establishing boundaries for enhanced control. For example:

LLM Prompting and Embedding: These tools allow developers to finely tune the bot’s tone and interaction style, ensuring a consistent and engaging user experience. Additionally, they enable the infusion of specialized knowledge content, such as information from an FAQ document or general support materials, directly into the bot’s devised responses.

Legacy Features: Complementing the advanced LLM capabilities, legacy features like suggested replies and pre-scripted answers provide a robust mechanism for directing the bot’s AI creativity. They offer a way to steer conversations along desired paths or, when necessary, bypass the LLM’s generative capabilities altogether to deliver foolproof, accurate answers.

This combination of innovative LLM features and reliable traditional tools empowers developers to create bots that are not only intelligent and dynamic but also dependable and aligned with specific conversational goals.

Overall, our Llama2 integration is not just about providing another conversational AI solution; it’s about offering a platform that can be molded and adapted to fit the unique identity and needs of each business, thereby enhancing both the developer experience and the end-user engagement.

ADM: The blog mentions expanding the proprietary LLM offering. Can you give examples of proprietary LLMs that you could integrate with in the future?

Kim: We’re incredibly excited about the burgeoning ecosystem of developers who are crafting specialized, proprietary LLMs. By fine-tuning existing open-source models like Llama2 or Falcon, these developers are achieving performance levels that often surpass established commercial LLMs such as GPT3.5 and Claude in specific use-cases.

Take cosmetics retail as an example: proprietary LLMs could be developed to offer highly personalized recommendations, considering factors like skin type, desired effect, and accompanying accessories. Such a degree of specialization and specificity in user experience is something that general-purpose LLM might struggle to match.

As we move forward, our commitment is to nurture this wave of innovation. We plan to enable seamless integrations of these proprietary LLMs into Sendbird’s bot interface. This will empower developers to deploy pioneering AI applications that not only meet but exceed market demands and user expectations.

ADM: What kind of businesses or industries do you see benefiting the most from the Llama2 integration?

Kim: The Llama2 integration is particularly advantageous for industries where data privacy and regulatory compliance are of paramount importance. The healthcare sector, for instance, often operates under stringent regulations that prohibit the dissemination of confidential patient information. Financial services face similar constraints when it comes to customer data.

The ability to run Llama2 privately ensures that these industries can leverage the power of conversational AI while remaining in full compliance with legal and ethical standards. This makes the integration a valuable asset for any business where data security and privacy are non-negotiable.

ADM: How does the Llama2 integration contribute to Sendbird’s vision for ‘Conversational AI’?

Kim: Our commitment is to nurture the wave of AI innovation in web and mobile communication. AI has already created a significant paradigm shift in our industry. Previously, the power of user-to-user conversations was primarily accessible to businesses with communities, dual-sided networks, or dedicated support staff. However, with the advent of large language models, we’re experiencing a transformative shift where generative AI can effectively substitute human interactions.

The integration of Llama2 is particularly aligned with this vision. Its high performance, coupled with a unique focus on privacy and security, makes it a crucial element of our offering. Llama2 enables online businesses to bridge gaps in the customer journey, particularly in areas where human support was previously unfeasible or reserved for high-value interactions. Take a small ecommerce website for example– it can now offer a recommendation and sales chatbot 24/7. Or, a real estate bot could effectively qualify a prospect before transferring it to a real sales agent.

Moreover, Llama2 addresses the significant challenges of ethics and compliance in AI. It ensures that customers with stringent privacy and ethical requirements can achieve their business goals without compromise. In essence, Llama2 is not just enhancing our AI capabilities; it’s ensuring that these advancements are not only accessible but secure and aligned with the highest ethical standards.

In short, Llama2 brings conversational AI to everyone

ADM: What are the security implications of integrating an open-source LLM like Llama2, and how is Sendbird addressing them?

Kim: Integrating an open-source LLM like Llama2 requires careful consideration of security measures to protect against vulnerabilities. Sendbird addresses these implications with its secure AWS cloud that implements robust security protocols, regularly updating and patching the system, and ensuring that data handling complies with industry standards. Our commitment to security is unwavering, and we continuously work to safeguard our platform and our users’ data.

ADM: Can you provide some use-cases or examples where Llama2’s capabilities can be uniquely leveraged within the Sendbird platform?

Kim: Llama2 can be uniquely leveraged in customer support, providing instant and accurate responses to queries. In e-commerce, it can offer personalized shopping assistance. In healthcare, it can provide preliminary medical advice. Its ability to understand and generate human-like responses makes it ideal for enhancing user engagement and closing gaps in the customer journey across various industries.

Additionally, because Sendbird handles clients’ conversational data, it can help clients progress to fine-tuning seamlessly using already hosted data. And it can happen securely within Sendbird’s ecosystem without any third party. To supercharge our clients in picking the right set of data to fine-tune, Sendbird offers both the backend and frontend support for features like feedbacks that are critical in evaluation.

ADM: What’s on the horizon for Sendbird’s AI-powered features? Any other partnerships or integrations we should be looking out for?

Kim: Sendbird is continuously exploring new partnerships and integrations to enhance our AI-powered features. We are looking into AI-powered analytics and recommendations for actions based on these analytics, real time AI-driven moderation in various languages, multichannel communication between AI chatbots and users, and integrations into various vertical platforms such as CRMs and marketing automation tools. Stay tuned for exciting updates and collaborations that will redefine the landscape of conversational AI.

About John S. Kim

John S. Kim is the Co-Founder and CEO of SendBird, the no. 1 conversations platform for mobile apps. The platform currently serves over 250M monthly chat users across the world’s leading companies, such as DoorDash, Reddit, Yahoo!, Rakuten, Paytm, and Walgreens. Under John’s leadership, Sendbird has raised $220M+ in funding from ICONIQ, Tiger Global, Meritech, Emergence, SoftBank Vision Fund, Wilab, Shasta, August, and Y Combinator. Sendbird officially became a unicorn in 2021. John is an entrepreneur, angel investor, and thought leader. He is a frequent contributor to publications such as Forbes, Fortune, TechCrunch, and more.