In this podcast with Sujit Mohanty, General Manager of Field Engineering at Databricks, learn how the recent achievement of FedRAMP high for Databricks on AWS creates impactful opportunities for modernization within the Federal Government
A New Era in Public Sector Data Intelligence
[Will Rose]
Welcome back to Carahcast, the podcast from Carahsoft Technology, the trusted government IT solutions provider. Subscribe to get the latest technology updates in public sector. My name is Will Rose, the sales director with Carisoft.
Joining me today is Suj Mahanty, the general manager for public sector field engineering at Databricks. And we're going to be talking about the importance of FedRAMP within the government market, and also through the lens of Databricks' recent FedRAMP high authorization on AWS. Suj, thanks for joining us today.
Absolutely, appreciate the time, Will. Thank you. Yeah, of course.
So for our listeners, could you please start by telling us a little bit about Databricks, and then a little bit about your role as the general manager for public sector field engineering.
[Sujit Mohanty]
Yeah, so Databricks is the data intelligence platform company. Started roughly 2013, 2014. Been focused on public sector since 2017, so over eight years.
Focused on delivering an analytics platform for the federal government, state and local education, as well as a lot of healthcare agencies as well, too. So really, we're kind of a leader on large-scale workloads for analytics, open source, open standards. And kind of my background was 20-plus years in public sector, currently running our pre-sale solutions architecture team.
The entire federal organization public sector is about 150 folks really dedicated to enabling key outcomes for our organization. But also focused on a number of other companies as well across the space, prior Microsoft, Pivotal, that were really key, shaping software, data analytics, and other capabilities for the federal government.
[Will Rose]
Yeah, awesome. Now, we're here today to talk about Databricks' recent announcements that you've received FedRAMP high authorization on AWS GovCloud, and also DoD ILL Level 5 on GovCloud as well. So maybe start by getting your quick thoughts on how Databricks is prioritizing security for our government customers.
[Sujit Mohanty]
Yeah, so security is probably one of the most important things, as we know, especially for our agencies in today's world, especially from an analytics standpoint. The idea of being able to share information, being able to have governance while still maintaining open standards, so you have a hardened core all the way to and through is extremely critical. So achieving FedRAMP authorization at pretty much all levels as possible is really, really key for us.
So recently we achieved our FedRAMP high at the early part of this year, and then also for the DoD Impact Level 5, we achieved that in November of 2024. So two key authorizations from a security standpoint for our end government customers. There's an aspect of being able to have US-based personnel that supports these environments, being able to have auditing capabilities, hardened core, so being able to have hardened enhanced security compliance features, hardened operating systems as required by the platforms.
But this becomes really key for a handful, and actually a number of big use cases for our customers, everything from national security use cases to data sharing, fraud, waste, and abuse, operational efficiency. So we're pretty excited by the opportunity to bring this platform from a SaaS capability in higher classification capabilities.
[Will Rose]
Yeah, absolutely, and you touched on it. Within FedRAMP there are different levels. You've got low for more publicly available information, moderate for something that may be more sensitive, but not always quite confidential.
And then there's high, which is information related to critical operations, national security, emergency services. So Databricks achieved FedRAMP high on AWS GovCloud. Of course, that's no small feat.
Could you talk about why getting that level of authorization is so significant, especially when it comes to supporting highly sensitive government workloads?
[Sujit Mohanty]
Yeah, so high really is extremely critical for those sensitive government workloads, but mainly around protecting unclassified information and really protecting things that are highly sensitive and also mission-critical information as well, too. So building upon everything that you have from a FedRAMP moderate perspective, which is pretty, I'll say relatively easy to get, but is achievable in commercial environments, you have to use things like GovCloud in order to have a jailed-off environment in order to achieve something like FedRAMP high. So there's a lot of engineering that goes into that, but the level of information, the level of use cases that you can support becomes fully more expanded as you go forward.
So one of the key areas, things like public safety, national security use cases, as I talked about a little bit before, too, being able to actually do information sharing amongst really sensitive data sets fully opens up, especially for things like law enforcement capabilities and public-private partnerships as you go forward as well, too.
[Will Rose]
Yeah, makes sense. And so let's talk a little bit more about specifics. Achieving FedRAMP high is going to empower teams to take advantage of the platform and advanced analytics, machine learning, all while having confidence in its security and ability to scale.
But what's that going to open up for federal agencies from a perspective of their data strategies?
[Sujit Mohanty]
Yeah, I think there's a key aspect of organizations are now trying to figure out how they can, and I kind of mentioned this a bit ago, how they can actually share their information securely amongst one another. So there's the aspect of being able to take these sensitive data sets that typically live in siloed environments and be able to share them with other organizations or share them internally, but also being able to apply proper governance as required while still supporting open standards. Those are sort of like two key primitives as far as what we've built Databricks on, from the whole starting from Apache Spark all the way through what we do for Delta, Iceberg support from a storage standpoint, and now with our Unity Catalog solution, which is key for governance.
Those are two of the key primitives that for us from a commercial standpoint and a government standpoint, were requirements inside of our platform. And so the use cases that we've been able to ensure for our customers, when I talk about national security, being able to do things like operational efficiency as well, those become some key use cases we can support.
[Will Rose]
Yeah, absolutely, and we at Carisoft, we've been working with Databricks probably the last seven to eight years, and so we saw some of those early adopters of the platform and the early use cases, but maybe go into a bit more depth to the degree that you can about some of the examples where agencies are using Databricks today, either boosting national security efforts, detecting fraud, waste and abuse, things of that nature.
[Sujit Mohanty]
Yeah, absolutely. So we're working with roughly over 400 public sector organizations across federal, across state and local, over 80% of the executive departments in the U.S. federal government, so it's everything from really enabling things from a machine learning standpoint, but some of the key use cases we're doing across some of our core customers, USCIS, for instance, is really leveraging Databricks, especially at Department of Homeland Security, for really streamlining the path to citizenship, so the entire immigration process.
Many aspects of that are powered via our analytics platform. Center for Medicare and Medicaid, so at CMS, for instance, being able to do fraud, waste and abuse, and tracking that, things from not just COVID-specific related type programs, but also from an implementation from a healthcare standpoint, being able to really do deep analytics tied to that for fraud, waste and abuse. And then the other side would be for U.S. Food and Drug Administration as well, being able to see things like drug interactions or fraudulent drugs as well that are coming potentially to market by potentially nefarious folks, that is an aspect of the FDA, being able to do that level of analytics. And then probably when I mentioned national security once or twice, the aspect of being able to support DOD type use cases, so one of our biggest use cases is actually the Advana analytics platform, so we're actually doing and supporting, it's over 3,000 systems, 100,000 end users, and that becomes like a really powerful force of gravity, really being able to aggregate so much information across the DOD, and really what was powering it was the DOD audit.
[Will Rose]
Yeah, and it's neat with that particular program you just mentioned and that when they started using it for one use case and then they realized how powerful it was, well they could replicate it to a lot of others as well.
[Sujit Mohanty]
Yeah, absolutely. And I think that's where that unlock for DOD IL-5 becomes really, really critical, being able to have mission critical information and have your governance tied to it, controlling who's able to interact with that and then being able to securely share it not only internally but also mission partners as well, becomes very powerful for that level of compliancy.
[Will Rose]
Yeah, and really at the heart of this conversation today is around security and security is at the heart of everything in public sector and FedRAMP of course really reinforces that and that's why it's so important, but within the Databricks platform, what are some of those standout security and compliance features and capabilities that government teams should know about that'll give them the confidence in using it for sensitive work?
[Sujit Mohanty]
Yeah, probably one of the two key areas, and I mentioned this two or three times, but one of the biggest ones is actually Unity Catalog. So inside of our Databricks platform or the Databricks Intelligence platform, ultimately Unity Catalog as far as having a standard open standards catalog, it's open source, but being able to provide governance is one of the most important things and it's not just governance inside of our platform, it's also external object shares, external resources as well. That's a critical thing as far as having governance and being able to control who can access what and how you can securely share things with external parties as well in that entire life cycle, but that builds upon open standards and so being able to support open storage formats is one of the most critical key things and so we actually support the two biggest industry formats right now tied to it, Delta Lake and also Iceberg as well. So those are uniquely shaped inside the platform from an open source standpoint. So you got open standards and then you got governance on top of that.
So those become kind of the underpinnings from a security standpoint and then there's like other feature sets that you need for higher classification regions ultimately. So things like customer managed keys, so allowing bring your own key from our government customers to be able to bring that into their environment and maintain their entire key life cycle, key rotation, that becomes pretty key. Being able to have private link as well, so being able to have a secure connection between their agency back in with their cloud resources and the platform, that's extremely critical.
Two or three other key things, I mentioned like a hardened security profile so that's not just at the operating system level but also the compensating controls as well. That becomes critical but then also serverless security as well. So being able to, in today's world of SaaS, being able to provide capabilities like independent of the customer having to worry about the management of infrastructure and doing that in a serverless fashion but providing the egress controls from a network security standpoint with that as well, that becomes a very powerful combo.
So those five, six combinations tie back within an open catalog at the end of the day. Open and secure catalog and open and secure storage formats becomes super powerful.
[Will Rose]
Yeah, absolutely. And your point on kind of this navigation of a customer moving towards a cloud is a good kind of segue to my next question which is maybe let's talk a little bit more about the migration journey itself. So we've both been in this industry for a while, we've seen the government making these shifts towards leveraging cloud and SaaS solutions but it can be a big leap.
So especially with all those compliance requirements that federal agencies face, how is Databricks supporting agencies through that onboarding process itself while keeping them in line with regulation and compliance every step of the way?
[Sujit Mohanty]
So when we think of workloads that an organization will want to move, one of the biggest ones that we see inside of our federal customers and really a lot of state and local customers as well too is actually legacy enterprise data warehouses. They exist everywhere. Everything from the Exadata through SaaS, through Teradata, there's so many different or on-prem appliances like Netezza, they exist in pretty much every agency and many are in some form of an end-of-life requirement.
So you've got either an on-prem or potentially something that may have been kind of forklifted to the cloud temporarily but then they really want to be modernized and pulled securely over. So it's a combination of being able to do an assessment as far as the amount of workload that has to happen, the security compliance requirements that are tied to that workload, and then being able to break it down to the actual migration steps. So analysis is just as important as the planning of the workload steps as well.
So there's a combination of tools that we can do. We have a very strong professional services team. Also have four deployed engineers that can come and actually help our customers in order to actually work directly on-site or work directly from a virtual standpoint to take some of those on-prem workloads and pull them into our platform inside of a SaaS capability.
But then there's also the tooling that we have as well. We acquired a company called Bleybridge recently and they were really focused on analyzing legacy workloads and being able to quantify how much work is required to then be able to port them over to a modern platform like Databricks. So it's that combination of tooling and also capability that exists that allows us to be able to securely and safely do that for our customers.
[Will Rose]
And all in the service of making sure that it's done right the first time the customer can adopt the platform and gets the value out of it.
[Sujit Mohanty]
Yeah, absolutely. And that's one of the most key things is usually the driver becomes some form of a business outcome type for our customers. Whether it's fraud, waste, and abuse, operational efficiency, maybe it could be a cyber-type use case, but they may be using legacy platforms in order to achieve those outcomes.
And so being able to help them actually gain agility by moving to a more modern analytics platform allows them to do a lot of unlock as far as being able to share information and really achieve those agency imperatives as they go forward.
[Will Rose]
Yeah, makes sense. So we're coming close to an end here, and so my last question for you, Suj, so achieving FedRAMP high is a major milestone. We've discussed it pretty thoroughly here, but what's next?
What do public sector teams have to look forward to from Databricks and any new capabilities or expanded support in the pipeline today?
[Sujit Mohanty]
Yeah, so I kind of mentioned that we've been around since 2017 with our Databricks Federal LLC and really focused on public sector. I think some of the areas, you know, that commitment that we have to the government already eight years in, going to be coming up on a decade here as we go forward, but there's many more things that we're heavily focused on, brand new capabilities. One of the key things I'm pretty excited about is bringing some of our Gen AI capabilities to these higher classification regions.
Customers are asking for that. There's a lot more that you've got to do than just simply taking your data and interfacing with a large language model. You've got to be careful what knowledge base you're informing.
How do you securely do that? How do you actually be able to pivot between different types of models? Maybe you want to train your own model depending on the use case maybe for national security.
So we have a whole slew of capabilities that are coming into our classification regions at not only just FedRAMP moderate, but eventually FedRAMP high as well and beyond that are tied around Gen AI, which are going to be really, really powerful as well. And then the other aspect is being able to go serverless all the way through the entire stack. That's one of those things from a modern SaaS platform capability.
Really helps unlock not just features but agility for our customers. So being able to have that at FedRAMP moderate, FedRAMP high becomes really powerful capabilities for our customers going forward. But really I think the one we're ultimately proud of with having the platform on FedRAMP high and also for DoD IL5 is having Unity catalog already available today.
So being able to provide that aspect of open standards governance that is missing for many agencies, they can get started right now with that.
[Will Rose]
Yeah, very cool. Well, exciting stuff ahead.
[Sujit Mohanty]
Absolutely.
[Will Rose]
Sujai, I think that's all that I've got. Thank you very much for your time.
[Sujit Mohanty]
Appreciate it, Will.