#![no_std]

#![warn(missing_docs)]

#![allow(unused_imports)]

#![allow(clippy::too_many_arguments)]

#![allow(clippy::transmute_ptr_to_ptr)]

#![cfg_attr(docsrs, feature(doc_cfg))]

//! A crate that safely exposes arch intrinsics via `#[cfg()]`.

//!

//! `safe_arch` lets you safely use CPU intrinsics. Those things in the

//! [`core::arch`](core::arch) modules. It works purely via `#[cfg()]` and

//! compile time CPU feature declaration. If you want to check for a feature at

//! runtime and then call an intrinsic or use a fallback path based on that then

//! this crate is sadly not for you.

//!

//! SIMD register types are "newtype'd" so that better trait impls can be given

//! to them, but the inner value is a `pub` field so feel free to just grab it

//! out if you need to. Trait impls of the newtypes include: `Default` (zeroed),

//! `From`/`Into` of appropriate data types, and appropriate operator

//! overloading.

//!

//! * Most intrinsics (like addition and multiplication) are totally safe to use

//!   as long as the CPU feature is available. In this case, what you get is 1:1

//!   with the actual intrinsic.

//! * Some intrinsics take a pointer of an assumed minimum alignment and

//!   validity span. For these, the `safe_arch` function takes a reference of an

//!   appropriate type to uphold safety.

//!   * Try the [bytemuck](https://docs.rs/bytemuck) crate (and turn on the

//!     `bytemuck` feature of this crate) if you want help safely casting

//!     between reference types.

//! * Some intrinsics are not safe unless you're _very_ careful about how you

//!   use them, such as the streaming operations requiring you to use them in

//!   combination with an appropriate memory fence. Those operations aren't

//!   exposed here.

//! * Some intrinsics mess with the processor state, such as changing the

//!   floating point flags, saving and loading special register state, and so

//!   on. LLVM doesn't really support you messing with that within a high level

//!   language, so those operations aren't exposed here. Use assembly or

//!   something if you want to do that.

//!

//! ## Naming Conventions

//! The `safe_arch` crate does not simply use the "official" names for each

//! intrinsic, because the official names are generally poor. Instead, the

//! operations have been given better names that makes things hopefully easier

//! to understand then you're reading the code.

//!

//! For a full explanation of the naming used, see the [Naming

//! Conventions](crate::naming_conventions) page.

//!

//! ## Current Support

//! * `x86` / `x86_64` (Intel, AMD, etc)

//!   * 128-bit: `sse`, `sse2`, `sse3`, `ssse3`, `sse4.1`, `sse4.2`

//!   * 256-bit: `avx`, `avx2`

//!   * Other: `adx`, `aes`, `bmi1`, `bmi2`, `fma`, `lzcnt`, `pclmulqdq`,

//!     `popcnt`, `rdrand`, `rdseed`

//!

//! ## Compile Time CPU Target Features

//!

//! At the time of me writing this, Rust enables the `sse` and `sse2` CPU

//! features by default for all `i686` (x86) and `x86_64` builds. Those CPU

//! features are built into the design of `x86_64`, and you'd need a _super_ old

//! `x86` CPU for it to not support at least `sse` and `sse2`, so they're a safe

//! bet for the language to enable all the time. In fact, because the standard

//! library is compiled with them enabled, simply trying to _disable_ those

//! features would actually cause ABI issues and fill your program with UB

//! ([link][rustc_docs]).

//!

//! If you want additional CPU features available at compile time you'll have to

//! enable them with an additional arg to `rustc`. For a feature named `name`

//! you pass `-C target-feature=+name`, such as `-C target-feature=+sse3` for

//! `sse3`.

//!

//! You can alternately enable _all_ target features of the current CPU with `-C

//! target-cpu=native`. This is primarily of use if you're building a program

//! you'll only run on your own system.

//!

//! It's sometimes hard to know if your target platform will support a given

//! feature set, but the [Steam Hardware Survey][steam-survey] is generally

//! taken as a guide to what you can expect people to have available. If you

//! click "Other Settings" it'll expand into a list of CPU target features and

//! how common they are. These days, it seems that `sse3` can be safely assumed,

//! and `ssse3`, `sse4.1`, and `sse4.2` are pretty safe bets as well. The stuff

//! above 128-bit isn't as common yet, give it another few years.

//!

//! **Please note that executing a program on a CPU that doesn't support the

//! target features it was compiles for is Undefined Behavior.**

//!

//! Currently, Rust doesn't actually support an easy way for you to check that a

//! feature enabled at compile time is _actually_ available at runtime. There is

//! the "[feature_detected][feature_detected]" family of macros, but if you

//! enable a feature they will evaluate to a constant `true` instead of actually

//! deferring the check for the feature to runtime. This means that, if you

//! _did_ want a check at the start of your program, to confirm that all the

//! assumed features are present and error out when the assumptions don't hold,

//! you can't use that macro. You gotta use CPUID and check manually. rip.

//! Hopefully we can make that process easier in a future version of this crate.

//!

//! [steam-survey]:

//! https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam

//! [feature_detected]:

//! https://doc.rust-lang.org/std/index.html?search=feature_detected

//! [rustc_docs]: https://doc.rust-lang.org/rustc/targets/known-issues.html

//!

//! ### A Note On Working With Cfg

//!

//! There's two main ways to use `cfg`:

//! * Via an attribute placed on an item, block, or expression:

//!   * `#[cfg(debug_assertions)] println!("hello");`

//! * Via a macro used within an expression position:

//!   * `if cfg!(debug_assertions) { println!("hello"); }`

//!

//! The difference might seem small but it's actually very important:

//! * The attribute form will include code or not _before_ deciding if all the

//!   items named and so forth really exist or not. This means that code that is

//!   configured via attribute can safely name things that don't always exist as

//!   long as the things they name do exist whenever that code is configured

//!   into the build.

//! * The macro form will include the configured code _no matter what_, and then

//!   the macro resolves to a constant `true` or `false` and the compiler uses

//!   dead code elimination to cut out the path not taken.

//!

//! This crate uses `cfg` via the attribute, so the functions it exposes don't

//! exist at all when the appropriate CPU target features aren't enabled.

//! Accordingly, if you plan to call this crate or not depending on what

//! features are enabled in the build you'll also need to control your use of

//! this crate via cfg attribute, not cfg macro.

use core::{

  convert::AsRef,

  fmt::{Binary, Debug, Display, LowerExp, LowerHex, Octal, UpperExp, UpperHex},

  ops::{Add, AddAssign, BitAnd, BitAndAssign, BitOr, BitOrAssign, BitXor, BitXorAssign, Div, DivAssign, Mul, MulAssign, Neg, Not, Sub, SubAssign},

};

pub mod naming_conventions;

/// Turns a round operator token to the correct constant value.

#[macro_export]

#[cfg_attr(docsrs, doc(cfg(target_feature = "avx")))]

// Note(Lokathor): keep this at the crate root.

macro_rules! round_op {

  (Nearest) => {{

    #[cfg(target_arch = "x86")]

    use ::core::arch::x86::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_NEAREST_INT};

    #[cfg(target_arch = "x86_64")]

    use ::core::arch::x86_64::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_NEAREST_INT};

    _MM_FROUND_NO_EXC | _MM_FROUND_TO_NEAREST_INT

  }};

  (NegInf) => {{

    #[cfg(target_arch = "x86")]

    use ::core::arch::x86::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_NEG_INF};

    #[cfg(target_arch = "x86_64")]

    use ::core::arch::x86_64::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_NEG_INF};

    _MM_FROUND_NO_EXC | _MM_FROUND_TO_NEG_INF

  }};

  (PosInf) => {{

    #[cfg(target_arch = "x86")]

    use ::core::arch::x86::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_POS_INF};

    #[cfg(target_arch = "x86_64")]

    use ::core::arch::x86_64::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_POS_INF};

    _MM_FROUND_NO_EXC | _MM_FROUND_TO_POS_INF

  }};

  (Zero) => {{

    #[cfg(target_arch = "x86")]

    use ::core::arch::x86::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_ZERO, _mm256_round_pd};

    #[cfg(target_arch = "x86_64")]

    use ::core::arch::x86_64::{_MM_FROUND_NO_EXC, _MM_FROUND_TO_ZERO, _mm256_round_pd};

    _MM_FROUND_NO_EXC | _MM_FROUND_TO_ZERO

  }};

}

/// Declares a private mod and then a glob `use` with the visibility specified.

macro_rules! submodule {

  ($v:vis $name:ident) => {

    mod $name;

    $v use $name::*;

  };

  ($v:vis $name:ident { $($content:tt)* }) => {

    mod $name { $($content)* }

    $v use $name::*;

  };

}

// Note(Lokathor): Stupid as it sounds, we need to put the imports here at the

// crate root because the arch-specific macros that we define in our inner

// modules are actually "scoped" to also be at the crate root. We want the

// rustdoc generation of the macros to "see" these imports so that the docs link

// over to the `core::arch` module correctly.

// https://github.com/rust-lang/rust/issues/72243

#[cfg(target_arch = "x86")]

use core::arch::x86::*;

#[cfg(target_arch = "x86_64")]

use core::arch::x86_64::*;

#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]

submodule!(pub x86_x64 {

  //! Types and functions for safe `x86` / `x86_64` intrinsic usage.

  //!

  //! `x86_64` is essentially a superset of `x86`, so we just lump it all into

  //! one module. Anything not available on `x86` simply won't be in the build

  //! on that arch.

  use super::*;

  submodule!(pub m128_);

  submodule!(pub m128d_);

  submodule!(pub m128i_);

  submodule!(pub m256_);

  submodule!(pub m256d_);

  submodule!(pub m256i_);

  submodule!(pub m512_);

  submodule!(pub m512d_);

  submodule!(pub m512i_);

  // Note(Lokathor): We only include these sub-modules with the actual functions

  // if the feature is enabled. Ae *also* have a cfg attribute on the inside of

  // the modules as a "double-verification" of sorts. Technically either way on

  // its own would also be fine.

  // These CPU features follow a fairly clear and strict progression that's easy

  // to remember. Most of them offer a fair pile of new functions.

  #[cfg(target_feature = "sse")]

  submodule!(pub sse);

  #[cfg(target_feature = "sse2")]

  submodule!(pub sse2);

  #[cfg(target_feature = "sse3")]

  submodule!(pub sse3);

  #[cfg(target_feature = "ssse3")]

  submodule!(pub ssse3);

  #[cfg(target_feature = "sse4.1")]

  submodule!(pub sse4_1);

  #[cfg(target_feature = "sse4.2")]

  submodule!(pub sse4_2);

  #[cfg(target_feature = "avx")]

  submodule!(pub avx);

  #[cfg(target_feature = "avx2")]

  submodule!(pub avx2);

  #[cfg(target_feature = "avx512f")]

  submodule!(pub avx512);

  // These features aren't as easy to remember the progression of and they each

  // only add a small handful of functions.

  #[cfg(target_feature = "adx")]

  submodule!(pub adx);

  #[cfg(target_feature = "aes")]

  submodule!(pub aes);

  #[cfg(target_feature = "avx512ifma")]

  submodule!(pub avx512ifma);

  #[cfg(target_feature = "bmi1")]

  submodule!(pub bmi1);

  #[cfg(target_feature = "bmi2")]

  submodule!(pub bmi2);

  #[cfg(target_feature = "fma")]

  submodule!(pub fma);

  #[cfg(target_feature = "lzcnt")]

  submodule!(pub lzcnt);

  #[cfg(target_feature = "pclmulqdq")]

  submodule!(pub pclmulqdq);

  #[cfg(target_feature = "popcnt")]

  submodule!(pub popcnt);

  #[cfg(target_feature = "rdrand")]

  submodule!(pub rdrand);

  #[cfg(target_feature = "rdseed")]

  submodule!(pub rdseed);

  /// Reads the CPU's timestamp counter value.

  ///

  /// This is a monotonically increasing time-stamp that goes up every clock

  /// cycle of the CPU. However, since modern CPUs are variable clock rate

  /// depending on demand this can't actually be used for telling the time. It

  /// also does _not_ fully serialize all operations, so previous instructions

  /// might still be in progress when this reads the timestamp.

  ///

  /// * **Intrinsic:** `_rdtsc`

  /// * **Assembly:** `rdtsc`

  pub fn read_timestamp_counter() -> u64 {

    // Note(Lokathor): This was changed from i64 to u64 at some point, but

    // everyone ever was already casting this value to `u64` so crater didn't

    // even consider it a problem. We will follow suit.

    #[allow(clippy::unnecessary_cast)]

    unsafe { _rdtsc() as u64 }

  }

  /// Reads the CPU's timestamp counter value and store the processor signature.

  ///

  /// This works similar to [`read_timestamp_counter`] with two main

  /// differences:

  /// * It and also stores the `IA32_TSC_AUX MSR` value to the reference given.

  /// * It waits on all previous instructions to finish before reading the

  ///   timestamp (though it doesn't prevent other instructions from starting).

  ///

  /// As with `read_timestamp_counter`, you can't actually use this to tell the

  /// time.

  ///

  /// * **Intrinsic:** `__rdtscp`

  /// * **Assembly:** `rdtscp`

  pub fn read_timestamp_counter_p(aux: &mut u32) -> u64 {

    unsafe { __rdtscp(aux) }

  }

  /// Swap the bytes of the given 32-bit value.

  ///

  /// ```

  /// # use safe_arch::*;

  /// assert_eq!(byte_swap_i32(0x0A123456), 0x5634120A);

  /// ```

  /// * **Intrinsic:** `_bswap`

  /// * **Assembly:** `bswap r32`

  pub fn byte_swap_i32(i: i32) -> i32 {

    #[allow(unused_unsafe)]

    unsafe { _bswap(i)}

  }

  /// Swap the bytes of the given 64-bit value.

  ///

  /// ```

  /// # use safe_arch::*;

  /// assert_eq!(byte_swap_i64(0x0A123456_789ABC01), 0x01BC9A78_5634120A);

  /// ```

  /// * **Intrinsic:** `_bswap64`

  /// * **Assembly:** `bswap r64`

  #[cfg(target_arch="x86_64")]

  pub fn byte_swap_i64(i: i64) -> i64 {

    #[allow(unused_unsafe)]

    unsafe { _bswap64(i) }

  }

});