Associativity of ##, double constants and preprocessor tokens

You might one day be confronted to the need to compose double constants by using the preprocessor. This is a tricky affair, since already the first naive try like this doesn’t work:

#define FRACTIONAL_WRONG(FRAC) .FRAC

Why is that so? For the preprocessor the dot and the following parameter are separate tokens. Thus called e.g as FRACTIONAL_WRONG(1) something like ‘. 1’ would be produced a stray dot followed by a blank and a number. This is nowhere a valid token sequence for the C compiler. And obviously the following macro, meant to produce a fractional number is wrong for the same reasons:

#define FRACTION_WRONG(INT, FRAC) INT.FRAC

Ok, we all know, to glue together tokens there is the
## operator in the preprocessor. The following actually
works:

#define FRACTIONAL(FRAC) . ## FRAC
#define __FRACTION(INT, FRAC) INT ## FRAC
#define _FRACTION(INT, FRAC) __FRACTION(INT, FRAC)
#define FRACTION(INT, FRAC) _FRACTION(INT, FRACTIONAL(FRAC))

/* using it */
#define INTEGERPART 4
#define FRACTIONALPART 01
static double a = FRACTION(INTEGERPART, FRACTIONALPART);

But we will see below that this is somehow just be coincidence.

Let us now try to generalize our idea to produce general doubles, including an exponent. One could be tempted to try something like this:

#define EXPONENT_WRONG(ESIGN, EXP) E ## ESIGN ## EXP
#define __DOUBLE_WRONG(SIGN, PN, EXP) SIGN PN ## EXP
#define _DOUBLE_WRONG(SIGN, PN, EXP) __DOUBLE_WRONG(SIGN, PN, EXP)
#define DOUBLE_WRONG(SIGN, INT, FRAC, ESIGN, EXP) _DOUBLE_WRONG(SIGN, FRACTION(INT, FRAC), EXPONENT_WRONG(ESIGN, EXP))

That is, we would try to first write an analogous macro that composes the exponent and then try to combine the two parts into one global macro. For this seemingly innocent declaration

static double b = DOUBLE_WRONG(-, 4, 01, +, 5);

My preprocessor says something weird like

error_paste.c:27:1: error: pasting "E" and "+" does not give a valid preprocessing token
error_paste.c:27:1: error: pasting "+" and "5" does not give a valid preprocessing token

And yours should say something similar, if it is standard compliant. The problem is that a preprocessor token that starts with an alphabetic character may only contain alphanumeric characters (plus underscore). Our example for FRACTIONAL only worked, because by chance a `dot’ followed by numbers is a valid token by itself, namely a floating point number.

A more direct approach would be to have a macro that pastes 6 tokens together

#define PASTE6_NOTSOGOOD(a, b, c, d, e, f) a ## b ## c ## d ## e ## f

and then hoping that something like the following would work:

#define DOUBLE_NOTSOGOOD(SIGN, INT, FRAC, ESIGN, EXP) SIGN PASTE6(INT, ., FRAC, E, ESIGN, EXP)

static double b = DOUBLE_NOTSOGOOD(-, 4, 01, +, 5);

An for most preprocessors it would: glued together from left to right each intermediate step would always consist of a valid preprocessor token. The actual rules of the preprocessor that allow for this are a bit more complicated, but basically in addition to alphanumeric tokens all starting parts of double constants (without prefix sign) are valid preprocessor tokens. ouff…

… you think. But there is a last subtlety which is the associativity of the ## operator. It is not specified whether or not it is from left to right. If we fall upon one that does it from right to left, we are screwed. So if we want to be portable, we have to go even further.

#define PASTE2(a, b) a ## b
#define _PASTE2(a, b) PASTE2(a, b)
#define PASTE3(a, b, c) _PASTE2(PASTE2(a, b), c)
#define PASTE4(a, b, c, d) _PASTE2(PASTE3(a, b, c), d)
#define PASTE5(a, b, c, d, e) _PASTE2(PASTE4(a, b, c, d), e)
#define PASTE6(a, b, c, d, e, f) _PASTE2(PASTE5(a, b, c, d, e), f)

static double b = PASTE6(4, ., 01, E, +, 7);
Advertisements

Detect empty macro arguments

The macro NARG2 that we introduced in post still has a major disadvantage, it will not be able to detect an empty argument list. This is due to a fundamental difference between C and its preprocessor. For C a parenthesis () is empty and contains no argument. For the preprocessor it contains just one argument, and this argument is the empty token.

So in fact NARG2 is cheating. It doesn’t count the number of arguments that it receives, but returns the number of commas plus one. In particular, even if it receives an empty argument list it will return 1. The following two macros better expresses this property:

#define _ARG16(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, ...) _15
#define HAS_COMMA(...) _ARG16(__VA_ARGS__, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0)

As before, these are constrained to a maximum number of arguments (here 16), but you will easily work out how to extend it to larger maximum values.

Now, when we want to write a macro that detects an empty argument we will be using the feature that a function macro that is not followed by an open parenthesis will be left alone. This we will do with the following macro that just transforms the following parenthesis and contents into a comma.

#define _TRIGGER_PARENTHESIS_(...) ,

The idea is to put the arguments that we want to test between the macro and its parenthesis, such that the macro only triggers if the arguments are empty:

_TRIGGER_PARENTHESIS_ __VA_ARGS__ (/*empty*/)

To do so we, must first check for some corner cases where special properties of the argument might mimic the behavior that we want to test.

#define ISEMPTY(...)                                                    \
_ISEMPTY(                                                               \
          /* test if there is just one argument, eventually an empty    \
             one */                                                     \
          HAS_COMMA(__VA_ARGS__),                                       \
          /* test if _TRIGGER_PARENTHESIS_ together with the argument   \
             adds a comma */                                            \
          HAS_COMMA(_TRIGGER_PARENTHESIS_ __VA_ARGS__),                 \
          /* test if the argument together with a parenthesis           \
             adds a comma */                                            \
          HAS_COMMA(__VA_ARGS__ (/*empty*/)),                           \
          /* test if placing it between _TRIGGER_PARENTHESIS_ and the   \
             parenthesis adds a comma */                                \
          HAS_COMMA(_TRIGGER_PARENTHESIS_ __VA_ARGS__ (/*empty*/))      \
          )

Here we distinguish four different cases, of which the last is just the main idea as exposed above. The first helps to exclude the trivial case, that __VA_ARGS__ already contains a comma by itself. The two others test if the two possible combinations of _TRIGGER_PARENTHESIS_, __VA_ARGS__ and (/*empty*/) also already trigger a comma to their output.

Now the outcome of this will be calling the macro _ISEMPTY with four different 0-1-values according to different cases that __VA_ARGS__ presents. In particular, the case that __VA_ARGS__ is empty corresponds exactly to the outcome _ISEMPTY(0, 0, 0, 1). All other outcomes will indicate that it was non-empty. We will detect this case with the following helper macro.

#define _IS_EMPTY_CASE_0001 ,

and we leave all the other 15 cases undefined. Now with

#define PASTE5(_0, _1, _2, _3, _4) _0 ## _1 ## _2 ## _3 ## _4
#define _ISEMPTY(_0, _1, _2, _3) HAS_COMMA(PASTE5(_IS_EMPTY_CASE_, _0, _1, _2, _3))

we will exactly detect the case we are interested in.

As a test here comes all of that together in a block. This is not a reasonable C program but just something to run through the preprocessor to test the validity of the approach.

#define _ARG16(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, ...) _15
#define HAS_COMMA(...) _ARG16(__VA_ARGS__, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0)
#define _TRIGGER_PARENTHESIS_(...) ,

#define ISEMPTY(...)                                                    \
_ISEMPTY(                                                               \
          /* test if there is just one argument, eventually an empty    \
             one */                                                     \
          HAS_COMMA(__VA_ARGS__),                                       \
          /* test if _TRIGGER_PARENTHESIS_ together with the argument   \
             adds a comma */                                            \
          HAS_COMMA(_TRIGGER_PARENTHESIS_ __VA_ARGS__),                 \
          /* test if the argument together with a parenthesis           \
             adds a comma */                                            \
          HAS_COMMA(__VA_ARGS__ (/*empty*/)),                           \
          /* test if placing it between _TRIGGER_PARENTHESIS_ and the   \
             parenthesis adds a comma */                                \
          HAS_COMMA(_TRIGGER_PARENTHESIS_ __VA_ARGS__ (/*empty*/))      \
          )

#define PASTE5(_0, _1, _2, _3, _4) _0 ## _1 ## _2 ## _3 ## _4
#define _ISEMPTY(_0, _1, _2, _3) HAS_COMMA(PASTE5(_IS_EMPTY_CASE_, _0, _1, _2, _3))
#define _IS_EMPTY_CASE_0001 ,

#define EATER0(...)
#define EATER1(...) ,
#define EATER2(...) (/*empty*/)
#define EATER3(...) (/*empty*/),
#define EATER4(...) EATER1
#define EATER5(...) EATER2
#define MAC0() ()
#define MAC1(x) ()
#define MACV(...) ()
#define MAC2(x,y) whatever
ISEMPTY()
ISEMPTY(/*comment*/)
ISEMPTY(a)
ISEMPTY(a, b)
ISEMPTY(a, b, c)
ISEMPTY(a, b, c, d)
ISEMPTY(a, b, c, d, e)
ISEMPTY((void))
ISEMPTY((void), b, c, d)
ISEMPTY(_TRIGGER_PARENTHESIS_)
ISEMPTY(EATER0)
ISEMPTY(EATER1)
ISEMPTY(EATER2)
ISEMPTY(EATER3)
ISEMPTY(EATER4)
ISEMPTY(MAC0)
ISEMPTY(MAC1)
ISEMPTY(MACV)
/* This one will fail because MAC2 is not called correctly */
ISEMPTY(MAC2)

Edit: What is presented here is a version with minor improvement to capture the case of MAC0. Notice the restriction on the argument of ISEMPTY that is apparent with MAC2. In fact ISEMPTY should work when it is called with macros as argument that expect 0, 1 or a variable list of arguments. If called with a macro X as an argument that itself expects more than one argument (such as MAC2) the expansion leads to an invalid use of that macro X.

Default arguments for C99

Provide a way to have default arguments to functions in C99.

From C++ we know the concept of default arguments for functions. Briefly stated, this concept allows to call a function with less arguments than it is specified and provide default values for the missing ones. As an example lets take the following prototype:

      void one_or_two(int a, int b = 5);

Here one_or_two may be called with one or two arguments and in the case of one, the value 5 is provided.

The goal here is to have the same effect in C, C99 to be precise, by using the macro preprocessor and inline functions. To do that will use the following elements/features

  1. Macros that hide a function
  2. Counting macro arguments
  3. Choose the right version of a function call
  4. inline functions for default arguments

Macros that hide a function

An important property of the C preprocessor is its lack of recursion. At a first glance it looks that this is merely a restriction and not an advantage. But here the fact that using a macro name inside the expansion of that same macro is just left `as is’ is quite helpful. But even more than that, a macro that is defined to receive arguments, but that is found without following parenthesis, is also left alone. The global view of our macro will be something like the following:

#define one_or_two(...) ONE_OR_TWO_ARGS(one_or_two, __VA_ARGS__)

So here, when calling the macro in a program the macro definition will be inserted and the token one_or_two which is found there will be left as such. Then, the remaining part of the arguments is expanded and if ONE_OR_TWO_ARGS is itself a macro, it will be expanded.

Here we also use a feature that is only normalized since C99, variable macro arguments. This is indicated by the ... in the definition of one_or_two. By that, one_or_two may be called with any number of arguments and the token __VA_ARGS__ in the definition is replaced by the arguments, including the commas between them.

Counting macro arguments

To implement the macro ONE_OR_TWO_ARGS we will need to determine how many arguments it receives. This can be achieved with something like the following:

#define _ARG2(_0, _1, _2, ...) _2
#define NARG2(...) _ARG2(__VA_ARGS__, 2, 1, 0)

If NARG2 is called with two arguments the 2 of its expansion is in third position and we will see this 2. If it is called with just one argument the 1 will be in that place and thus be the result of the expansion. You probably easily imagine an extension of that macro to treat say 64 or 128 arguments.

Choose the right version of a function call

Finally we want two different versions of ONE_OR_TWO_ARGS, one if it is called with one argument and one for two:

#define _ONE_OR_TWO_ARGS_1(NAME, a) a, NAME ## _default_arg_1()
#define _ONE_OR_TWO_ARGS_2(NAME, a, b) a, b

Both macros receive in NAME the name of the function that this all about. But only _ONE_OR_TWO_ARGS_1 uses it to produce the name of a default function to call. If NAME would be one_or_two the function call produced by the preprocessor would be one_or_two_default_arg_1(). This is achieved by the ## operator of the preprocessor that glues two adjacent tokens into one.

A mechanism to choose between the two now could look like this

#define __ONE_OR_TWO_ARGS(NAME, N, ...) _ONE_OR_TWO_ARGS_ ## N (NAME, __VA_ARGS__)
#define _ONE_OR_TWO_ARGS(NAME, N, ...) __ONE_OR_TWO_ARGS(NAME, N, __VA_ARGS__)
#define ONE_OR_TWO_ARGS(NAME, ...) NAME(_ONE_OR_TWO_ARGS(NAME, NARG2(__VA_ARGS__), __VA_ARGS__))

__ONE_OR_TWO_ARGS is supposed to receive N the actual number of arguments in __VA_ARGS__ as it second argument and constructs a call to either _ONE_OR_TWO_ARGS_1 or _ONE_OR_TWO_ARGS_2.

_ONE_OR_TWO_ARGS is just an intermediate step that ensures that all arguments are evaluated sufficiently often such that really the value of the call to NARG2 is put in place. Otherwise _ONE_OR_TWO_ARGS_ and NARG2 would be glued into a single (sensless) token _ONE_OR_TWO_ARGS_NARG2.

Finally, ONE_OR_TWO_ARGS places the name of the function, determines the number of arguments it has received and calls _ONE_OR_TWO_ARGS.

inline functions for default arguments

Remains to define the function for the default argument. It might look like the following:

static inline
int one_or_two_default_arg_1(void) {  return 5; }

Here the inline keyword (stolen from C++ and new in C99) suggests that the function body should be `substituted’ in place where a call to it appears. So at a first glance it looks that this defines something similar to a macro, but actually an important difference is the point where evaluation takes place; an inline function itself is evaluated in the context in which it is defined. Only its arguments are evaluated at the place it is called. On the other hand, a macro is not evaluated at the point of its definition but at the point of its call.

To see that take the case that a default function is not just evaluating a constant expression but doing some real work.

static inline
int inline_counter_default_arg_1(void) { ++myCounter, return myCounter; }
#define macro_counter_default_arg_1(void) (++myCounter)

Here inline_counter_default_arg_1 uses the global counter variable myCounter that is visible at the point of its definition. If there is none, this results in an error. macro_counter_default_arg_1 evaluates myCounter in the context of the caller, and this might actually refer to different variables at different places. The first results in the rule which C++ implements for default arguments: they are evaluated in the scope of definition. The second one is a different model for which C++ has no equivalent.

Finally a complete example.

#include <stdio.h>
#define _ARG2(_0, _1, _2, ...) _2
#define NARG2(...) _ARG2(__VA_ARGS__, 2, 1, 0)
#define _ONE_OR_TWO_ARGS_1(NAME, a) a, NAME ## _default_arg_1()
#define _ONE_OR_TWO_ARGS_2(NAME, a, b) a, b
#define __ONE_OR_TWO_ARGS(NAME, N, ...) _ONE_OR_TWO_ARGS_ ## N (NAME, __VA_ARGS__)
#define _ONE_OR_TWO_ARGS(NAME, N, ...) __ONE_OR_TWO_ARGS(NAME, N, __VA_ARGS__)
#define ONE_OR_TWO_ARGS(NAME, ...) NAME(_ONE_OR_TWO_ARGS(NAME, NARG2(__VA_ARGS__), __VA_ARGS__))
#define one_or_two(...) ONE_OR_TWO_ARGS(one_or_two, __VA_ARGS__)

// function definition, also calls the macro, but you wouldn't notice
void one_or_two(int a, int b) { printf("%s seeing a=%d and b=%d\n", __func__, a, b); }

static inline int one_or_two_default_arg_1(void) {  return 5; }

int main (void) {
  // call with default argument
  one_or_two(6);
  // call with default argument
  one_or_two(6, 0);
  // taking a function pointer still works
  void (*func_pointer)(int, int) = one_or_two;
  // But this pointer may only be called with the complete set of
  // arguments
  func_pointer(3, 4);
}

Right shift on signed types is not well defined

The shift operators (<< and >>) shift the bits in a word to the left or the right. From such an explanation it doesn’t follow directly what should happen with the bits at the word boundaries. There are several commonly used strategies

  • logical:
    Bits that go beyond the word boundary are dropped and the new positions are filled with zeroes.
  • ones:
    Bits that go beyond the word boundary are dropped and the new positions are filled with ones.
  • arithmetic:
    1. Shift is `logical’ for positive values.
    2. For negative values right shift is `ones’ and
    3. left shift is `logical’ but always sets the highest order bit (sign bit) to 1.
  • circular: Bits that go beyond the word boundary are reinserted at the other end.

`Arithmetic’ shift has its name from the fact that it implements an integer multiplication or division by a power of two.

For unsigned integer types C prescribes that the shift operators are `logical’ . So e.g (~0U >> 1) results in a word of all ones but for the highest order bit which is 0. The picture darkens when it comes to signed types. Here the compiler implementor may choose between a `logical’ and an `arithmetic’ shift. Basically this means that the use of the right shift operator on signed values is not portable unless very special care is taken. We can detect which shift is implemented by the simple expression ((~0 >> 1) < 0)

  • If the shift is `logical’ the highest order bit of the left side of the comparison is 0 so the result is positive.
  • If the shift is `arithmetic’ the highest order bit of the left side is 1 so the result is negative.

Observe in particular that in case of an arithmetic shift (~0 >> 1) == ~0. So this operator has two fixed points in that case, 0 and -1. If we want a portable shift we may choose the following operations

#define LOGSHIFTR(x,c) (((x) >> (c)) &amp; ~(~0 << (sizeof(int)*CHAR_BIT - (c)))

This produces a mask with the correct number of 1’s in the low order bits and performs a bitwise and with the result of the compiler shift. Observe

  • This supposes that x is of type int, a type independent definition would be much more complicated.
  • c is evaluated twice so don’t use side effects here.

Here is a C99 program to test your compiler.

#include <limits.h>
#include <stdio.h>

extern
int logshiftr(int x, unsigned c);

extern
int arishiftr(int x, unsigned c);

#define HIGHONES(c) ((signed)(~(unsigned)0 << (sizeof(signed)*CHAR_BIT - (c))))
#define HIGHZEROS(c) (~HIGHONES(c))

inline
int logshiftr(int x, unsigned c) {
  return (x >> c) &amp; HIGHZEROS(c);
}

inline
int arishiftr(int x, unsigned c) {
  return logshiftr(x, c) ^ (x < 0 ? HIGHONES(c) : 0);
}

int main(int argc) {
  int b = argc > 1 ? argc : 0;
  int val[11u] = { b, b + 1, b - 1, b + 2, b - 2, b + 3, b - 3, b + 4, b - 4, b + 5, b - 5};
  printf("shift\tvalue\t>>\tlogical\tarith\n");
  for (unsigned sh = 1; sh < 3; ++sh)
    for (unsigned i = 0; i < 11u; ++i)
      printf("%+d\t%+d\t%+d\t%+d\t%+d\n",
             sh,
             val[i],
             (val[i] >> sh),
             logshiftr(val[i], sh),
             arishiftr(val[i], sh));
}

How many bits has a byte?

I recently stumbled about this seemingly silly question when trying to write a C macro that depends on the width of a type.

So everybody knows the short answer, 8, as is also expressed in the commonly used French word for byte: `octet’. But surprisingly for me the long answer is more complicated than that: it depends on the historical context, the norms that you apply, and even then you have to dig a lot of text to come to it.

C has a definition of the type char, and the language specification basically uses the terms char and  byte interchangeably.

Historically, in times there have been platforms with chars (and thus bytes) that had a width different from 8, in particular some early computers coded printable characters with only 6 bits and had a word size of 36. And later other constructors found it more convenient to have words of 16 bits to be the least addressable unit. C90 didn’t wanted to exclude such platforms and just stated

The number of bits in a char is defined in the macro CHAR_BIT
CHAR_BIT can be any value but must be at least 8

and even C99 still just states:

A byte contains CHAR_BIT bits, and the values of type unsigned char range from 0 to (2^CHAR_BIT) – 1.

But then, on the page for the include file stdint.h it states

The typedef name int N _t designates a signed integer type with width N, no  padding  bits,  and  a  two’s-complement representation. Thus, int8_t denotes a signed integer type with a width of exactly 8 bits.

So far so good, if there is an int8_t we can deduce that sizeof(int8_t) must be 1 and CHAR_BIT must be 8. But then the POSIX standard says

The following types are required:
int8_t
int16_t
int32_t
uint8_t
uint16_t
uint32_t

Which forces CHAR_BIT to be 8, and basically also implies that at least for small width types the representation must be two’s-complement on any POSIX compatible platform.

More reading:

Some forum discussion
The POSIX specification of stdint.h limits.h