Native code interoperability¶

Scala Native provides an interop layer that makes it easy to interact with foreign native code. This includes C and other languages that can expose APIs via C ABI (e.g. C++, D, Rust etc.)

All of the interop APIs discussed here are defined in scala.scalanative.native package. For brevity, we’re going to refer to that namespace as just native.

Extern objects¶

Extern objects are simple wrapper objects that demarcate scopes where methods are treated as their native C ABI-friendly counterparts. They are roughly analogous to header files with top-level function declarations in C.

For example, to call C’s malloc one might declare it as following:

@native.extern
object libc {
  def malloc(size: native.CSize): native.Ptr[Byte] = native.extern
}

native.extern on the right hand side of the method definition signifies that the body of the method is defined elsewhere in a native library that is available on the library path (see Linking with native libraries). The signature of the external function must match the signature of the original C function (see Finding the right signature).

Finding the right signature¶

To find a correct signature for a given C function one must provide an equivalent Scala type for each of the arguments:

C Type	Scala Type
`void`	`Unit`
`bool`	`native.CBool`
`char`	`native.CChar`
`signed char`	`native.CSignedChar`
`unsigned char`	`native.CUnsignedChar` [1]
`short`	`native.CShort`
`unsigned short`	`native.CUnsignedShort` [1]
`int`	`native.CInt`
`long int`	`native.CLongInt`
`unsigned int`	`native.CUnsignedInt` [1]
`unsigned long int`	`native.CUnsignedLongInt` [1]
`long`	`native.CLong`
`unsigned long`	`native.CUnsignedLong` [1]
`long long`	`native.CLongLong`
`unsigned long long`	`native.CUnsignedLongLong` [1]
`size_t`	`native.CSize`
`ptrdiff_t`	`native.CPtrDiff` [2]
`wchar_t`	`native.CWideChar`
`char16_t`	`native.CChar16`
`char32_t`	`native.CChar32`
`float`	`native.CFloat`
`double`	`native.CDouble`
`void*`	`native.Ptr[Byte]` [2]
`int*`	`native.Ptr[native.CInt]` [2]
`char*`	`native.CString` [2] [3]
`int (*)(int)`	`native.CFunctionPtr1[native.CInt, native.CInt]` [2] [4]
`struct { int x, y; }*`	`native.Ptr[native.CStruct2[native.CInt, native.CInt]]` [2] [5]
`struct { int x, y; }`	Not supported

[1]	See Unsigned integer types.

[2]	See Pointer types.

[3]	See Byte strings.

[4]	See Function pointers.

[5]	See Memory layout types.

Linking with native libraries¶

C compilers typically require to pass an additional -l mylib flag to dynamically link with a library. In Scala Native, one can annotate libraries to link with using the @native.link annotation.

@native.link("mylib")
@native.extern
object mylib {
  def f(): Unit = native.extern
}

Whenever any of the members of mylib object are reachable, the Scala Native linker will automatically link with the corresponding native library.

As in C, library names are specified without the lib prefix. For example, the library libuv corresponds to @native.link("uv") in Scala Native.

It is possible to rename functions using the @name annotation. Its use is recommended to enforce the Scala naming conventions in bindings:

import scala.scalanative.native._
@link("uv")
@extern
object uv {
  @name("uv_uptime")
  def uptime(result: Ptr[CDouble]): Int = extern
}

If a library has multiple components, you could split the bindings into separate objects as it is permitted to use the same @link annotation more than once.

Variadic functions¶

One can declare variadic functions like printf using native.CVararg auxiliary type:

@native.extern
object stdio {
  def printf(format: native.CString,
             args: native.CVararg*): native.CInt = native.extern
}

Pointer types¶

Scala Native provides a built-in equivalent of C’s pointers via native.Ptr[T] data type. Under the hood pointers are implemented using unmanaged machine pointers.

Operations on pointers are closely related to their C counterparts and are compiled into equivalent machine code:

Operation	C syntax	Scala Syntax
Load value	`*ptr`	`!ptr`
Store value	`*ptr = value`	`!ptr = value`
Pointer to index	`ptr + i`, `&ptr[i]`	`ptr + i`
Elements between	`ptr1 - ptr2`	`ptr1 - ptr2`
Load at index	`ptr[i]`	`ptr(i)`
Store at index	`ptr[i] = value`	`ptr(i) = value`
Pointer to field	`&ptr->name`	`ptr._N`
Load a field	`ptr->name`	`!ptr._N`
Store a field	`ptr->name = value`	`!ptr._N = value`

Where N is the index of the field name in the struct. See Memory layout types for details.

Function pointers¶

It is possible to use external functions that take function pointers. For example given the following signature in C:

void test(void (* f)(char *));

One can declare it as following in Scala Native:

def test(f: CFunctionPtr1[CString, Unit]): Unit = native.extern

To pass a Scala function to CFunctionPtrN, you need to use the conversion function CFunctionPtr.fromFunctionN():

def f(s: CString): Unit = ???
def g(): Unit = test(CFunctionPtr.fromFunction1(f))

Memory management¶

Unlike standard Scala objects that are managed automatically by the underlying runtime system, one has to be extra careful when working with unmanaged memory.

Zone allocation. (since 0.3)

Zones (also known as memory regions/contexts) are a technique for semi-automatic memory management. Using them one can bind allocations to a temporary scope in the program and the zone allocator will automatically clean them up for you as soon as execution goes out of it:
```
native.Zone { implicit z =>
  val buffer = native.alloc[Byte](n)
}
```
Zone allocation is the preferred way to allocate temporary unmanaged memory. It’s idiomatic to use implicit zone parameters to abstract over code that has to zone allocate.

One typical example of this are C strings that are created from Scala strings using native.fromCString. The conversion takes implicit zone parameter and allocates the result in that zone.

When using zone allocated memory one has to be careful not to capture this memory beyond the lifetime of the zone. Dereferencing zone-allocated memory after the end of the zone is undefined behavior.
Stack allocation.

Scala Native provides a built-in way to perform stack allocations of using native.stackalloc function:
```
val buffer = native.stackalloc[Byte](256)
```
This code will allocate 256 bytes that are going to be available until the enclosing method returns. Number of elements to be allocated is optional and defaults to 1 otherwise.

When using stack allocated memory one has to be careful not to capture this memory beyond the lifetime of the method. Dereferencing stack allocated memory after the method’s execution has completed is undefined behavior.
Manual heap allocation.

Scala Native’s library contains a bindings for a subset of the standard libc functionality. This includes the trio of malloc, realloc and free functions that are defined in native.stdlib extern object.

Calling those will let you allocate memory using system’s standard dynamic memory allocator. Every single manual allocation must also be freed manually as soon as it’s not needed any longer.

Apart from the standard system allocator one might also bind to plethora of 3-rd party allocators such as jemalloc to serve the same purpose.

Undefined behavior¶

Similarly to their C counter-parts, behavior of operations that access memory is subject to undefined behaviour for following conditions:

Dereferencing null.
Out-of-bounds memory access.
Use-after-free.
Use-after-return.
Double-free, invalid free.

Memory layout types¶

Memory layout types are auxiliary types that let one specify memory layout of unmanaged memory. They are meant to be used purely in combination with native pointers and do not have a corresponding first-class values backing them.

native.Ptr[native.CStructN[T1, ..., TN]]

Pointer to a C struct with up to 22 fields. Type parameters are the types of corresponding fields. One may access fields of the struct using _N helper methods on a pointer value:
```
val ptr = native.stackalloc[native.CStruct2[Int, Int]]
!ptr._1 = 10
!ptr._2 = 20
println(s"first ${!ptr._1}, second ${!ptr._2}")
```
Here _N computes a derived pointer that corresponds to memory occupied by field number N.
native.Ptr[native.CArray[T, N]]

Pointer to a C array with statically-known length N. Length is encoded as a type-level natural number. Natural numbers are types that are composed of base naturals Nat._0, ... Nat._9 and an additional Nat.Digit constructor. So for example number 1024 is going to be encoded as following:
```
import scalanative.native._, Nat._

type _1024 = Digit[_1, Digit[_0, Digit[_2, _4]]]
```
Once you have a natural for the length, it can be used as an array length:
```
val ptr = native.stackalloc[CArray[Byte, _1024]]
```
Addresses of the first twenty two elements are accessible via _N accessors. The rest are accessible via ptr._1 + index.

Byte strings¶

Scala Native supports byte strings via c"..." string interpolator that gets compiled down to pointers to statically-allocated zero-terminated strings (similarly to C):

import scalanative.native._

// CString is an alias for Ptr[CChar]
val msg: CString = c"Hello, world!"
stdio.printf(msg)

Additionally, we also expose two helper functions native.toCString and native.fromCString to convert between C-style and Java-style strings.

Unchecked casts¶

Quite often, C interfaces expect the user to perform unchecked casts to convert between different pointer types, or between pointers and integer values. For this particular use case, we provide obj.cast[T] that is defined in the implicit class native.CCast. Unlike Scala’s asInstanceOf, cast does not provide any safety guarantees.

Platform-specific types¶

Scala Native defines the type Word and its unsigned counterpart, UWord. A word corresponds to Int on 32-bit architectures and to Long on 64-bit ones.

Size of types¶

In order to statically determine the size of a type, you can use the sizeof function which is Scala Native’s counterpart of the eponymous C operator. It returns the size in bytes:

println(sizeof[Byte])    // 1
println(sizeof[CBool])   // 1
println(sizeof[CShort])  // 2
println(sizeof[CInt])    // 4
println(sizeof[CLong])   // 8

It can also be used to obtain the size of a structure:

type TwoBytes = CStruct2[Byte, Byte]
println(sizeof[TwoBytes])  // 2

Unsigned integer types¶

Scala Native provides support for four unsigned integer types:

native.UByte
native.UShort
native.UInt
native.ULong

They share the same primitive operations as signed integer types. Primitive operation between two integer values are supported only if they have the same signedness (they must both signed or both unsigned.)

Conversions between signed and unsigned integers must be done explicitly using signed.toUByte, signed.toUShort, signed.toUInt, signed.toULong and conversely unsigned.toByte, unsigned.toShort, unsigned.toInt, unsigned.toLong.

Continue to Libraries.