32 wide - only AMD has a 64 wide mode

The create is called portable_simd.

There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width.

But the design and documentation of portable_simd makes the fixed size syntactically easy/the default and the width agnostic code harder.

Sure but there's no real way to use that in a portable way, at least not a way that maximises performance on every CPU you run it on. That's pretty much impossible at the moment.

The tentative plan is to open source all the compiler and `std` bits with our products built on top (compilers are not good businesses). More about our products coming in the next couple of months!

The goal of this work is to run existing unmodified CPU libraries (which may use core::simd) on the GPU. If you are manually writing ML-shaped workloads, it doesn't add any value over writing with tech like torch/tensorflow/jax which are custom built for those use-cases (except maybe familiarity if you are a CPU programmer).

Didn't want to go into crazy detail in the post.

Each family of operations is a trait parameterized by the operation itself:

  pub trait EvaluateReduction<Operation, T>: LaneEvaluator {
      /// Reduce one distributed definition to an ordinary uniform scalar.
      fn evaluate_reduction(&self, value: LaneValue<Self, role::Distributed, T>) -> T;
  }

  let one   = evaluator.splat::<Splat, _>(1_u32);
  let two   = evaluator.splat::<Splat, _>(2_u32);
  let three = evaluator.binary::<Add, _>(one, two);

  let total   = evaluator.reduce::<Sum, u32>(three);   // a uniform u32
  let running = <Executor as EvaluateScan<Scan<Sum, Exclusive>, u32>>::scan(&evaluator, three);

As mentioned in the post, execution shape is typed too. A static shuffle takes its control as a type-level constant, and the shuffle mode constrains which controls are expressible:

  // Shift down one lane, keeping our own value where the source is inactive.
  let down  = <Executor as EvaluateShuffle<Shuffle<Down>, DownOrSelf<1>, u32>>::shuffle(&ev, v);
  // Broadcast from lane zero.
  let bcast = <Executor as EvaluateShuffle<Shuffle<Broadcast>, WarpLane<0>, u32>>::shuffle(&ev, down);
  // Butterfly exchange with the neighbor one bit away.
  let bfly  = <Executor as EvaluateShuffle<Shuffle<Xor>, Butterfly<1>, u32>>::shuffle(&ev, bcast);

  <ScopedWarpExecutor<'_, WarpUniform> as EvaluateBarrier<Barrier<Device>>>::barrier(evaluator)
  // error[E0277]: the trait bound `Device: NvptxBarrierScope` is not satisfied
  //               help: the trait `NvptxBarrierScope` is implemented for `Warp`

  // Six work items across four active lanes: two chunks, based at 0 and 4.
  <Executor as EvaluateStripMine<StripMine, (WorkItems, ActiveLanes<StripMined<4>>), i32>>::
      for_each_strip_mined(
          &evaluator,
          (WorkItems::new(6)?, ActiveLanes::new(4)?),
          |index, active| {
           // ...
          },
      );

I'm confused too. How does this fit between these approaches for paraellization:

  - CUDA kernels and Tiles (e.g. Cudarc, cuda-oxide, rust-gpu etc) - SIMD on the GPU. (E.g. as in the title...)
  - CPU SIMD using avx or SSE instructions (And probably thin wrappers for vectors so you can have sane syntax). Or the maybe-upcoming core simd which should abstract over architecture-specific instructions. Magic floats etc which do 4-16 computations at once, but are a bit clumsy to work with
  - Rayon thread pools - arbitrary parallel computations, including SIMD, one per CPU core.

It is not currently available but we intend to make it available after we launch our products.

Part of our thesis is that decent GPUs are in every shipping device and most software doesn't use them and should.

The goal is to use similar abstractions and code across both the CPU and GPU where it makes sense.

It's just a way for us to add minutia and details that most don't care / need to know about. There are three audiences we try to make the posts accessible for: Rust people who don't know about GPUs, GPU people who don't know about Rust, and non-Rust non-GPU people. The toggle lets knowledgable readers go "wait, what about..." and hopefully the toggle answers it.

Could also just be trolls attracted by the Rust topic.