CIO / Head of Infrastructure
A supportable platform for AI demand, with an architecture and a cost curve that can be explained to the board.
AI / GPU · Platforms we sell and architect
AI & Accelerated Compute.
The AI programs we have scoped usually had enough GPUs and lacked an operating model. ModernOps architects accelerated compute as a whole system, from accelerator and interconnect to storage, orchestration, governance and the power bill.
AI / ML Platform Leader
Dependable training and inference capacity, standardized environments, and a faster path from experiment to production.
Research Computing / HPC Director
SLURM, Kubernetes, fast fabrics, parallel storage and queue pressure, handled as one architecture with a cloud overflow valve.
FinOps / Security Stakeholder
Isolated tenants, auditable usage, quotas that bind, and cost attributed to the team, project or grant that spent it.
The decision
Key design
considerations.
01 /
Training or inference.
The first question in any scoping call. Inference and retrieval workloads run on single nodes and mid-range GPUs; training clusters are a different capital class. Most of the savings come from sizing for the workload you run today, which is usually smaller than the one in the keynote demo.
02 /
Buy, rent or hybrid.
Cloud wins for burst, exploration and access to frontier accelerators the day they exist. Owned capacity wins for sustained, predictable demand over a three-to-five-year horizon, where utilization drives the unit cost below any hourly rate. The estates we design end up with both, under one operating model.
03 /
The GPU is rarely the bottleneck.
Accelerators starve behind slow storage, thin interconnect and clumsy data movement. The design covers the whole path: NVLink or InfiniBand fabric, parallel and object tiers, and the pipeline feeding them, or the expensive part sits idle.
04 /
Utilization sets the unit cost.
An idle flagship GPU is the most expensive space heater in the building. Scheduling, priority queues, preemption and fractional sharing through MIG or time-slicing turn stranded capacity into throughput.
05 /
Cost attribution.
Industry surveys rank unpredictable cost among the top AI infrastructure concerns. The remedy is structural: metering by job, project and owner, quotas and budget policy at request time, idle reclamation, and a monthly FinOps review that reads like a utility bill.
What we architect
Reference
architectures.
Pattern 01
Single-node GPU server designs balancing accelerator class, memory and cost for production inference, sized to the model and the request volume.
Pattern 02
Rafay-orchestrated catalogs where approved notebooks, SLURM queues, Kubernetes namespaces and inference endpoints launch in minutes inside policy guardrails: tenant isolation, role-based access, quotas and telemetry from user and job down to pod, node, GPU and cost center.
Pattern 03
Azure CycleCloud with SLURM federation so existing on-premises queues overflow to cloud capacity with portable containers and unchanged job scripts where the workload permits. Partner-documented deployments of this pattern have cut queue waits by 60 percent.
Pattern 04
Purpose-built Penguin Solutions clusters, factory-integrated and delivered with ICE ClusterWare, for the workloads whose demand curve justifies owning the floor, with optional Penguin on-site or remote cluster management.
Pattern 05
Fabric and storage designed together: NVLink, NVSwitch or InfiniBand interconnect feeding parallel scratch, persistent shares and object tiers. One recent design scoped 500 terabytes to a petabyte of tiered parallel storage behind a 36-node accelerated cluster. Deep storage architecture lives on the Enterprise Storage page.
Pattern 06
High-density power and cooling in ModernOps data center regions in Pennsylvania and Arizona, for racks beyond what your building can feed, with managed operations wrapped around the environment.
The platforms
Supported
platforms.
Accelerator selection across generations and profiles, matched to model size, memory footprint, interconnect sensitivity and economics, and procured through channel and OEM programs and we handle allocation and lead times for you.
Where cloud economics favor it, the Azure catalog also exposes AMD MI300X-class profiles, and we will say so.
GPU and HPC SKUs from A100 through the current frontier, CycleCloud for burst, and the storage tiers that feed them: Managed Lustre, Azure NetApp Files and Blob. Delivered with the Microsoft practice’s landing-zone and cost-guardrail discipline.
Design, factory integration, deployment and optional management of purpose-built private clusters, with ICE ClusterWare as the operating layer. Penguin’s own published references run at extraordinary scale, including expanding Meta’s Research SuperCluster from 10,000 to 16,000 GPUs without cluster downtime and a 1,032-GPU Blackwell deployment. When we bring an integration partner to a private-cluster build, this is the bench.
PlatformRafay
The orchestration and self-service layer across Kubernetes, SLURM, VMs, bare metal, Azure subscriptions, AWS accounts and existing on-premises clusters: one catalog, one policy framework, one program-level view across every substrate.
Also on the line card
DDN, VDURA and VAST for parallel and scale storage (see Enterprise Storage), Dell and HPE GPU-capable server platforms (see Servers & Virtualization Proxmox VE).
The old model vs ModernOps
The ModernOps
difference.
| Dimension | The old model | ModernOps |
|---|---|---|
| Provisioning | Ticket-driven builds, hand-configured environments | Policy-guarded notebooks, queues, clusters and endpoints from templates, in minutes |
| Capacity | A fixed local cluster or disconnected cloud accounts | Hybrid placement across cloud, private clusters and existing estates by demand, priority and cost |
| Platform design | GPU servers bought separately from storage and network | Accelerator, CPU, fabric, storage, scheduler, identity and data path designed as one system |
| Cost control | The bill arrives without attribution | Metering by job and owner, quotas, budget policy at request time, idle reclamation, FinOps cadence |
| Utilization | Node metrics with no workload context | Correlated telemetry: user, job, pod, node, GPU, model and cost center |
| Collaboration | Separate tools, credentials and admin silos | Federated identity, isolated tenants and projects, one governed catalog |
| Accountability | Reseller, integrator, cloud and operator handoffs | One team architects, sources, integrates, onboards workloads and operates |
The first 30 days
The first
30 days.
| Phase | What happens | What you hold at the end |
|---|---|---|
| Assess and map · Days 1 to 10 | Inventory AI, analytics and HPC workloads. Profile model sizes, GPU-hours, storage and data movement, queue behavior, identity and existing cloud agreements and capacity. | A workload and dependency map, bottleneck analysis, demand classification and a first cloud-versus-private placement hypothesis. |
| Architect and validate · Days 11 to 20 | Produce the target architecture across accelerators, fabric, storage, orchestration, identity, observability and cost controls. Select the pilot workload and success criteria. | A reference architecture, capacity and quota plan, implementation backlog and measurable success criteria. |
| Pilot the foundation · Days 21 to 30 | Stand up or validate a pilot landing zone, connect identity and baseline policies, publish one approved service template, instrument cost and utilization dashboards, run a representative workload where prerequisites allow. | A working foundation or validated pilot, an optimization baseline and a production roadmap. |
Scope control
Thirty days buys you the map, the architecture and a validated pilot. Production is a longer program: the documented arc runs about sixteen weeks, from landing zone through workload onboarding sprints to governed steady state.
Two paths, one team, plus a third
Engagement
models.
Consume it in the cloud, own it as a private cluster, or host it on our floor. Cloud burst for elasticity, Penguin-built capacity for the multi-year baseline, and GPU-ready colocation in PA and AZ with managed operations wrapped around it: monitoring, platform support and escalation structured to the engagement.
Proof
Design work
and references.
The submitted architecture.
ModernOps design workA 22-page hybrid design for a research consortium spanning Azure GPU and HPC, Rafay self-service orchestration and private-cluster capacity: federated identity, tenant isolation, FinOps controls and workloads from model training to classical-quantum simulation, submitted with sourcing for each line.
Partner track record, named as such.
Penguin Solutions referencesPenguin Solutions’ references include growing Meta’s Research SuperCluster from 10,000 to 16,000 GPUs without downtime. Our role is the architecture and the accountability line; theirs is the integration bench for private-cluster builds.
Ecosystem outcomes, labeled.
Partner-documented, reported by the ecosystemPartner-documented deployments of this orchestration model report a 38 percent reduction in steady-state spend through idle reclamation across an environment of 200-plus researchers in 14 labs. These are partner figures, cited as evidence for the operating model rather than as ModernOps delivery results.
PA + AZ
ModernOps data center regions with GPU-ready power and cooling
50+
Technology partners across the stack this page designs against
FAQ
Frequently asked
questions.
01 /Do we need frontier training GPUs, or is that overkill?
For the mid-market AI workloads we have scoped, usually yes. Inference and retrieval run well on single-node designs with mid-range GPUs. Training clusters are a different capital class, and the workload profile decides.
02 /When does buying GPUs beat renting cloud capacity?
When demand is sustained and predictable over a three-to-five-year horizon and utilization stays high. Below that, rent. Around the crossover, run hybrid: owned baseline, cloud burst.
03 /Can our existing SLURM cluster burst into the cloud?
Yes, that is a documented pattern: CycleCloud with SLURM federation so queues overflow to Azure with portable containers and unchanged job scripts where the workload permits.
04 /Can researchers or developers self-serve GPUs without losing control?
Yes. Approved notebooks, queues, namespaces and endpoints publish as catalog items behind policy: quotas, tenant isolation, role-based access and full usage attribution.
05 /How do we stop expensive GPUs sitting idle?
Scheduling and priority queues, preemption for the right jobs, fractional sharing through MIG or time-slicing where supported, and automated reclamation of idle allocations. Utilization is the metric that pays for the platform.
06 /How do we attribute cost to teams, projects or grants?
Tagging and metering at the workload level, quotas and budget policy enforced at request time, and a FinOps cadence reporting cost per job, per model run and per million tokens.
07 /Can our existing server room power a GPU rack?
Often it lacks the capacity: GPU-dense racks draw multiples of standard rack power and need cooling to match. Power and cooling are the first calculation in every design, and GPU-ready colocation in our PA and AZ regions is the fallback when the building falls short.
08 /What storage does AI need?
Parallel scratch for training throughput, persistent shares for home and datasets, object tiers for archive, on a fabric that keeps the accelerators fed, designed together with the storage practice.
09 /What are GPU lead times really like?
Variable enough to plan around: allocation programs, OEM queues and configuration choices all move the date. We work through channel allocation for owned hardware and use reservations or spot capacity in cloud while you wait.
10 /Who runs it after deployment?
Your team, enabled with documentation and training; ModernOps, with platform support, monitoring and escalation structured to the engagement; or Penguin’s optional on-site or remote cluster management for private builds. Pick per layer.
Direct to an engineer
Tell us about
the workload.
Training or inference, cloud or floor: whatever you have. We will place it and show the cost model.