|
|
Automated Sciences owns a number of high
performance x86 computers each with a large (32-128GB)
memory foot print we have used this current group of
computers in recent years to run specialized high
performance compute workloads for a couple of customers.
These customers are currently not using these
computers so we are offering these computers for short term
3-6 month rental at very low prices, just enough to cover
power, connectivity, additional RAM, and maintenance costs.
The
following table lists the available systems and basic specs
and monthly rental cost. We only have one
of each listed system available now.
By December we can upgrade the RAM in two additional Dual
Intel Xeon SP2 systems (last line in table) and make these
available for a 6 month rental.
| CPU |
GiDP/s |
GfpFMA/s |
Memory |
GB/sec |
AI Features |
$/Mo |
| Intel Xeon SP2 18 Core |
5600 |
1400 |
96GB |
90 |
VNNI |
$180 |
| AMD Ryzen 9600 6 Core |
3600 |
800 |
96GB |
60 |
VNNI,BF16 |
$150 |
| AMD ThreadRipper 32 Core |
1400 |
700 |
32GB |
60 |
AVX2 |
$80 |
| AMD Epyc 24 Core |
900 |
400 |
64GB |
60 |
AVX2 |
$90 |
| Intel Xeon Haswell 2x10 core |
1100 |
400 |
64&16GB |
60/30 |
AVX2 |
$90 |
| Dual Intel Xeon SP2 2x18 core |
9400 |
2200 |
96&32GB |
90/60 |
VNNI |
$250 |
In
the table GiDP/s is tested AI performance of each
server. This is the speed of the 8 bit
integer dot product operation in billion ops/sec which is the
key operation for all modern AI
inference.
GfpFMA/s (billion/sec) is the speed of the
single precision floating point fused multiply add operation
in billions ops/sec key to most high performance computing
algorithms and floating point AI inference. Note to get
standard GFlops for comparison double this number.
For the offering that supports BF16 (brain float
16 bit) the BF16 performance is twice that of the listed
GfpFMA/s for this system. All listed systems
though support rapid vector conversion to/from FP16 to FP32 to
facilitate memory efficient 16 bit floating point model
parameters.
VNNI (vector neural network instructions)
provide an optimized dot product operation for 8 and 16 bit
integers, creating 32 but sum of products without
conversion. VNNI capability infers AVX512
foundation instruction support which offers 512 bit vector
operations for all integer and floating point math and
conversion operations. AVX512 capability also
fills out the parts of AVX(2) that limited efficient
auto-vectorization. This allows modern compilers
to do a good job vectorizing tight loops even in older legacy
code.
The dual Intel Xeon systems will perform best if
running just one or two separate but potentially related
nueral network models. A larger one on CPU1 and smaller
one on CPU2. Each model should be bound by software to
its CPU. See our AI Server White
paper for more info on this.
On any of the listed Xeon SP2 systems we can upgrade the RAM to 192GB on the first CPU1 for $80/month extra but only with a 6 month rental term.
Storage
For storage we offer 3 alternatives: Each
system has a 500GB hard drive built in that starts out with a
wiped drive and Ubuntu 26.04 installed.
We have a SAN onsite that can provided up to 4TB
RAID 6 redundant storage at cost of
$20/TB/month.
The SAN access speed is up to 800MB/sec from the
Xeon SP2 based systems and 120MB/sec from the remaining
systems. I will remind though that the
RAID array is shared and limited to 500MB/sec so the data rate
for any individual system can vary greatly but will usually be
about 400MB/sec for large reads like loading model parameters.
Although the SAN isolates access from each AI
server we highly recommend that when you initialize your SAN
volume that it is encrypted to ensure that any
proprietary/private data remains so. All
listed systems support hardware encryption so the encryption
overhead is minimal.
Lastly all the systems can take a single NVMe
SSD that the customer must supply. We will
install for the rental period and return it to you in whatever
state you leave it in at the end of the rental
period.
Internet
Internet connectivity is physically shared
between all systems but logically isolated between customers
(a single customer can have multiple system in their VLAN) and
will be at least 100Mbit/sec. I will note that
except for model upload and potentially a database upload fast
internet is not usually necessary after initial
setup.
We will open a single internet port assigned by
us to each system. The intent is to run an ssh
server on this port. So all access is
solely through ssh. Using port redirection a
server can be run on any port of the rented AI server and this
can be accessed by the same or different port number on your
local system. So for example you could run a web
server on the AI server that listens on the standard port 80
and then you can redirect all traffic going to port 80 on your
local/cloud server through ssh to the web server running
on the rented AI server. We can provide
instructions to do this as necessary.
All these AI Servers can access any internet
content without restrictions.
Software
Each AI server comes with Ubuntu 2604 installed
including common system utility packages. You will be
given the sole and primary user password that also allows root
access. We can provide limited support for
Ubuntu on this system.
You are free to install any other software you
need. All the popular AI frameworks and most high
performance science and engineering applications are available
with instructions for Ubuntu install on Intel/AMD x86_64
CPU's. So setup/usage instructions and online
support info should be widely available for these AI servers
running Ubuntu.
If for some reason you must run another OS we
can facilitate installing this through temporary remote KVM
access but this will be for just for a few days to
enable remote install and setup. We cannot ensure
compatibility or provide any support for an alternate
OS.
Support
As this is being done as a side business we are
not providing any application/AI software support for the use
of these systems even for an additional price.
We simply do not have the resources or knowledge to support AI
frameworks or models. Fortunately the most
popular ones are well documented and known to run without
issue on x86 servers like these.
If you loose ssh access to your system we will
quickly look into why with your permission and restore access
ASAP. If not quickly possible though we will
have to reinstall the OS wiping its partition (we will leave
your home directory alone).
We will also fix any hardware problems ASAP but
this may take up to 3 days. These systems have
been fully tested and burned in using HPC/AI workloads and are
server grade so hardware issues should be rare.
Typical Usage
Just about any engineering or science tool can
be run on these servers as although a few years old they still
represent the state of the art in computing and have enough
RAM for all science and engineering applications and as noted
above do have access to fast high capacity SAN storage needed
to hold ancillary/image data needed by many science
applications. The Threadripper system
in the table above is particularly well suited for this with
its 32 cores each with a 512KB of L2 cache and 64MB of fast L3
Cache that most science and engineering software will size and
localize its per CPU task to.
All these systems represent a way to experiment with AI for
SMB purposes without spending a fortune buying or renting an
AI accelerator. They can do
everything that a modern AI accelerator can do, just
slower. This includes training new facts
into an existing LLM, See our AI
Server white paper for more information.
How much slower depends on chosen AI server and
the AI accelerator you are comparing against but typically
these servers are 10-100 times slower. This
low speed makes them inappropriate for interactive AI use but
very capable of being given a complex agentic or image
generation task and coming back hours later with results just
as good as an AI accelerator would provide.
With the large memory footprint 8-16 agentic
tasks can be done at the same time with minimal impact on
individual task performance, see our AI Server white paper for
details and why.
These servers are also valuable to evaluate how
useful a custom augmented AI is to your organization with a
small investment before deciding to purchase an AI
accelerator.
Lastly I do expect AI accelerators designed for
inference with large >64GB memory footprints to become
available at much lower prices than todays accelerators as
early as 2028. So renting one or more CPU systems
now represents a low cost way to achieve the capability you
need now and save many thousands $$$ by waiting a year+ for
these new purpose built systems.
We have written a white
paper that describes the performance differences,
technical details, and applications information.
Renting
|
860.886.8368 info@automatedsciences.com |