----- Original Message ----- From: "Paul Mensonides" <yg-boost-users@m.gmane.org>

"midnight2silverlake" <midnight2silverlake@yahoo.com> wrote in message news:asq7h1+r192@eGroups.com...

...

I have what looks like a perfect application for Boost/Preprocessor. I have 192 entries in a Factory Map a portion of which are show below. I'd like to generate the permutations of the three macro inputs, but can't figure out where to start. Any help would be appreciated.

#define MAP_ENTRY_MCR( aLang, aType, aAcc ) \ \ mMap[ CKey( aLang, aType, aAcc ) ] \ = CVarX< aLang, aType, aAcc >::Make;

MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_NONE ); MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_INIT ); MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_ANYTIME );

MAP_ENTRY_MCR( DCTlanguageC, DCT_FLOAT , DCT_NONE ); ...

Thanks, Jeff Flinn

Hi Jeff,

I want to make certain that I'm clear about what you're trying to do. You have 192 consecutive "MAP_ENTRY_MCR( ... )" with different permutions each time, right? How many different elements for each "field" do you have? (i.e. how many DCT_DOUBLE, DCT_FLOAT, etc. and how many DCT_NONE, DCT_INIT, etc.)

aLang(4) => DCTlanguageAda, DCTlanguageAdsim, DCTlanguageC , DCTlanguageFortran aType(8)=> DCT_DOUBLE, DCT_FLOAT , DCT_INT_16, DCT_INT_32, DCT_INT_8 , DCT_UINT_16, DCT_UINT_32, DCT_UINT_8 aAcc(3)=> DCT_INIT, DCT_ANYTIME, DCT_NONE This accounts for 96 MAP_ENTRY_MCR's. There will actually be a 4th macro argument: aDim(2)=> DCT_Vector, DCT_Scalar Which accounts for the 192 entries. The DCT_ constants are a combination of enums and integer defines. aLang and aDim are enum entries, with sequential values starting with 1. aType and aAcc are integer defines. aType is non-sequential, aAcc is sequential starting with 1. It is possible for any of these to become non-sequential in the future.

There are several ways to do this, so let me know if I'm clear on what you're doing, and I'll help you.

Paul Mensonides

Thanks for the help Paul Jeff Flinn Applied Dynamics, International

"Jeff Flinn" <midnight2silverlake@yahoo.com> wrote in message news:000c01c29f84$ae37b0e0$ea02a8c0@adi.com...

----- Original Message ----- From: "Paul Mensonides" <yg-boost-users@m.gmane.org>

...

"midnight2silverlake" <midnight2silverlake@yahoo.com> wrote in message news:asq7h1+r192@eGroups.com...

...

I have what looks like a perfect application for Boost/Preprocessor. I have 192 entries in a Factory Map a portion of which are show below. I'd like to generate the permutations of the three macro inputs, but can't figure out where to start. Any help would be appreciated.

#define MAP_ENTRY_MCR( aLang, aType, aAcc ) \ \ mMap[ CKey( aLang, aType, aAcc ) ] \ = CVarX< aLang, aType, aAcc >::Make;

MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_NONE ); MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_INIT ); MAP_ENTRY_MCR( DCTlanguageC, DCT_DOUBLE , DCT_ANYTIME );

MAP_ENTRY_MCR( DCTlanguageC, DCT_FLOAT , DCT_NONE ); ...

Thanks, Jeff Flinn

Hi Jeff,

I want to make certain that I'm clear about what you're trying to do. You have 192 consecutive "MAP_ENTRY_MCR( ... )" with different permutions each time, right? How many different elements for each "field" do you have? (i.e. how many DCT_DOUBLE, DCT_FLOAT, etc. and how many DCT_NONE, DCT_INIT, etc.)

aLang(4) => DCTlanguageAda, DCTlanguageAdsim, DCTlanguageC , DCTlanguageFortran aType(8)=> DCT_DOUBLE, DCT_FLOAT , DCT_INT_16, DCT_INT_32, DCT_INT_8 , DCT_UINT_16, DCT_UINT_32, DCT_UINT_8 aAcc(3)=> DCT_INIT, DCT_ANYTIME, DCT_NONE

This accounts for 96 MAP_ENTRY_MCR's. There will actually be a 4th macro argument:

aDim(2)=> DCT_Vector, DCT_Scalar

Which accounts for the 192 entries.

Okay, so what we have here is a full-scale permutation of these four groups of values. It doesn't matter if they are sequential or not, because we'll deal with them by name rather than by value. (To address the topic of this thread, you _could_ use BOOST_PP_REPEAT, but only for three dimensions, and it wouldn't be a good idea for other reasons.) Generally speaking, there are three good options. Only one of them is a good option if you plan to use an EDG front-end (such as Comeau or Intel) because the sheer size of such a cartesian product will take a while to generate. option #1: lists ---------------> #include <boost/preprocessor/list/for_each_product.hpp> #define A_LANG \ (DCTlanguageAda, (DCTlanguageAdsim, (DCTlanguageC, \ (DCTlanguageFortran, BOOST_PP_NIL)))) \ /**/ #define A_TYPE \ (DCT_DOUBLE, (DCT_FLOAT, (DCT_INT_16, (DCT_INT_32, \ (DCT_INT_8, (DCT_UINT_16, (DCT_UINT_32, \ (DCT_UINT_8, BOOST_PP_NIL)))))))) \ /**/ #define A_ACC \ (DCT_INIT, (DCT_ANYTIME, (DCT_NONE, BOOST_PP_NIL))) \ /**/ #define A_DIM (DCT_Vector, (DCT_Scalar, BOOST_PP_NIL)) #define PRODUCT(r, product) MAP_ENTRY_MCR product #define MAP_ENTRY_MCR( aLang, aType, aAcc, aDim ) \ mMap[ CKey( aLang, aType, aAcc ) ] \ = CVarX< aLang, aType, aAcc >::Make; \ /* aDim? */ BOOST_PP_LIST_FOR_EACH_PRODUCT( PRODUCT, 4, ( A_LANG, A_TYPE, A_ACC, A_DIM ) ) #undef A_LANG #undef A_TYPE #undef A_ACC #undef A_DIM #undef PRODUCT #undef MAP_ENTRY_MCR ------------------------- The second option requires the latest CVS sources of the PP Lib (i.e. post-1.29 release). It is a variant of the above, but with a preprocessor data type called a "sequence" that is a little simpler and more efficient that a "list." option #2: sequences ---------------> #include <boost/preprocessor/seq/elem.hpp> #include <boost/preprocessor/seq/for_each_product.hpp> #define A_LANG \ (DCTlanguageAda)(DCTlanguageAdsim) \ (DCTlanguageC)(DCTlanguageFortran) \ /**/ #define A_TYPE \ (DCT_DOUBLE)(DCT_FLOAT)(DCT_INT_16)(DCT_INT_32) \ (DCT_INT_8)(DCT_UINT_16)(DCT_UINT_32)(DCT_UINT_8) \ /**/ #define A_ACC \ (DCT_INIT)(DCT_ANYTIME)(DCT_NONE) \ /**/ #define A_DIM \ (DCT_Vector)(DCT_Scalar) \ /**/ #define PRODUCT(r, seq) \ MAP_ENTRY_MCR( \ BOOST_PP_SEQ_ELEM(0, seq), BOOST_PP_SEQ_ELEM(1, seq), \ BOOST_PP_SEQ_ELEM(2, seq), BOOST_PP_SEQ_ELEM(3, seq) \ ) \ /**/ #define MAP_ENTRY_MCR( aLang, aType, aAcc, aDim ) \ mMap[ CKey( aLang, aType, aAcc ) ] \ = CVarX< aLang, aType, aAcc >::Make; \ /* aDim? */ BOOST_PP_SEQ_FOR_EACH_PRODUCT( PRODUCT, (A_LANG)(A_TYPE)(A_ACC)(A_DIM) ) #undef A_LANG #undef A_TYPE #undef A_ACC #undef A_DIM #undef PRODUCT #undef MAP_ENTRY_MCR ------------------------- Both of the two options above use library primitives that are built for cartesian products such as this. However, they don't scale to this size well on slow preprocessors (e.g. EDG and maybe GCC). Luckily, we have a third alternative. However, this alternative is much more "invasive." It is a technique which I call "file iteration." In effect, it iterates over a file (or portion of a file) repeatedly with a different macro state each iteration. Furthermore, file iteration is supported up to five dimensions--we only have four here. I'll presume this stuff is in a header file called "map_entry_mcr.h". This psuedo-code roughly describes the implementation: for (int i = 0; i < aLang.size(); ++i) { for (int j = 0; j < aType.size(); ++j) { for (int k = 0; k < aAcc.size(); ++k) { for (int l = 0; l < aDim.size(); ++l) { MAP_ENTRY_MCR(i, j, k, l); } } } } option #3: file iteration ---------------> #if !BOOST_PP_IS_ITERATING #ifndef MAP_ENTRY_MCR_H // normal include guard #define MAP_ENTRY_MCR_H #include <boost/preprocessor/array/elem.hpp> #include <boost/preprocessor/array/size.hpp> #include <boost/preprocessor/iteration/iterate.hpp> // ----- normal file contents go here ----- // #define A_LANG \ (4, (DCTlanguageAda, DCTlanguageAdsim, \ DCTlanguageC, DCTlanguageFortran)) #define A_TYPE \ (8, ( \ DCT_DOUBLE, DCT_FLOAT, DCT_INT_16, DCT_INT_32, \ DCT_INT_8, DCT_UINT_16, DCT_UINT_32, DCT_UINT_8 \ )) #define A_ACC \ (3, (DCT_INIT, DCT_ANYTIME, DCT_NONE)) #define A_DIM \ (2, (DCT_Vector, DCT_Scalar)) #define ACCESS(array, frame) \ BOOST_PP_ARRAY_ELEM( \ BOOST_PP_FRAME_ITERATION(frame), \ array \ ) \ /**/ #define aLang ACCESS(A_LANG, 1) #define aType ACCESS(A_TYPE, 2) #define aAcc ACCESS(A_ACC, 3) #define aDim ACCESS(A_DIM, 4) // iterate over elements in "A_LANG" #define BOOST_PP_ITERATION_PARAMS_1 \ (3, (0, BOOST_PP_ARRAY_SIZE(A_LANG) - 1, "map_entry_mcr.h")) #include BOOST_PP_ITERATE() #undef A_LANG #undef A_TYPE #undef A_ACC #undef A_DIM #undef ACCESS #undef aLang #undef aType #undef aAcc #undef aDim // ----- other normal file contents go here ----- // #endif // EOF #elif BOOST_PP_ITERATION_DEPTH() == 1 // iterate over elements in "A_TYPE" #define BOOST_PP_ITERATION_PARAMS_2 \ (3, (0, BOOST_PP_ARRAY_SIZE(A_TYPE) - 1, "map_entry_mcr.h")) #include BOOST_PP_ITERATE() #elif BOOST_PP_ITERATION_DEPTH() == 2 // iterate over elements in "A_ACC" #define BOOST_PP_ITERATION_PARAMS_3 \ (3, (0, BOOST_PP_ARRAY_SIZE(A_ACC) - 1, "map_entry_mcr.h")) #include BOOST_PP_ITERATE() #elif BOOST_PP_ITERATION_DEPTH() == 3 // iterate over elements in "A_DIM" #define BOOST_PP_ITERATION_PARAMS_4 \ (3, (0, BOOST_PP_ARRAY_SIZE(A_DIM) - 1, "map_entry_mcr.h")) #include BOOST_PP_ITERATE() #else mMap[ CKey( aLang, aType, aAcc ) ] = CVarX< aLang, aType, aAcc >::Make; ? aDim #endif ------------------------- As you can see, this method is much more elaborate. The mechanism is primarily intended for repeating large swathes of code rather than cartesian products. Nevertheless, it works and is fairly fast--even on EDG. Make sure that you understand what is happening if you use this last example (maybe read the docs on file-iteration). Also, note that you can put all this iteration stuff in a separate file and merely include it where you want. The file iteration mechanism is a powerhouse, and this usage barely touches on its potential (which is partially why the implementation above is as obtuse as it is). The strengths of option #1 and option #2 is the locality of code. Option #3 is a lot more involved. The weaknesses of option #1 and option #2 is that it is really slow on EDG preprocessors and that all the code gets expanded on a single line and is completely undebuggable. Option #3 excels in this area--though in this case, the debuggability aspect is negligible for such a small snippet of source code. In any case, I urge you to read the documentation on each library primitive used and also the topic on file iteration. If you find anything unclear, then you can tell me, and I can improve the docs.

The DCT_ constants are a combination of enums and integer defines. aLang and aDim are enum entries, with sequential values starting with 1. aType and aAcc are integer defines. aType is non-sequential, aAcc is sequential starting with 1. It is possible for any of these to become non-sequential in

the future.

...

There are several ways to do this, so let me know if I'm clear on what you're doing, and I'll help you.

Paul Mensonides

Thanks for the help Paul

Jeff Flinn Applied Dynamics, International

Regards, Paul Mensonides

"Jeff Flinn" <midnight2silverlake@yahoo.com> wrote in message news:001101c2a04b$2d844630$ea02a8c0@adi.com...

Paul,

Awesome! Thanks for the extensive description and all your efforts. It seems like I run into this situation quite often. I'll be studying up on all of these primitives.

Thanks again, Jeff

Good luck. I hope, more than anything else, that it gives you an idea of what *can* be done with the PP lib, rather than just that this particular problem be solved. Paul Mensonides

----- Original Message ----- From: "Thomas Wenisch" <twenisch@ece.cmu.edu>

On Mon, 9 Dec 2002, Paul Mensonides wrote:

[ snip: lengthy example of 3 different ways to generate repeated code over a cartesian product of substitutions ]

These examples were very enlightening, thanks for going to the trouble of posting them to the list for all of us to see. I have a few questions:

No problem. Any questions about the Preprocessor Library from anyone here I'd be happy to answer. I am also happy to help anyone that needs it if I can--they don't call me the "PP Guru" for nothing, and if I can't (i.e. no time (or patience) ;)) I can still point you in the right direction.

1) Is there any reason to prefer lists over sequences on any compilers? It seems that the syntax for sequences is much nicer, since you don't have to spend time counting paranthesis.

Yes. A list has direct support for the nil state. I.e. 'BOOST_PP_NIL' is a nil list. Sequences, on the other hand, cannot be empty. In many application areas this doesn't matter, but in others the necessary "hacking around" that you have to do to deal with faked empty sequences makes it more worthwhile to use lists. Incidently, when/if C++ gets the preprocessor upgrades from C, a nil state will be directly supported. (C99 allows empty arguments.) Dealing with sequences is typically faster than lists and can easily replace tuples (e.g. (x, y, z)) for random data caches. I.e. element-wise sequence access is a *very* fast operation, even if the sequence is huge (supported up to 256, I believe). Lists cannot be effectively used this way--you'd bring Comeau C++ (for example) to a grinding halt. Also, appending to either the front or back of a sequence is as fast as you can get: #define SEQ (a)(b)(c) BOOST_PP_SEQ_ELEM(4, SEQ (x)(y)(z) SEQ) // y BOOST_PP_SEQ_ELEM(6, SEQ (x)(y)(z) SEQ) // a Sequences have many advantages over lists, but there is no such thing as an "empty" sequence--and that can be a major disadvantage. Therefore, both are supported (plus there is a lot of "legacy" code that uses lists) in the current CVS. Note that the 1.29 release does not contain the sequence implementation.

2) What is the data structure you use in the file iteration example (the structure which requires prepended lengths). Are the hardcoded lengths neccessary for file iteration to work, or could one of the other data structures (ie sequences) be used instead? I really like the simple syntax of the sequence type. Nothing to screw up :)

I assume that you mean this one --> (3, (a, b, c)). That data structure is called an "array," which is just an arbitrary name choice. It is, in effect, a high-level tuple that encodes its own size. The actual file iteration parameters are *required* to be arrays. The reason is that arrays and file-iteration existed before sequences, and I needed away to pass variable sized datasets into the iteration mechanism. I wanted the sample implementation earlier in this thread to automatically adjust if the size of the datasets increased. This meant I had to use a data type whose size I could calculate: array, sequence, list, but *not* tuple. Lists would be a *really* bad choice here because they a terrible choice for random access. Furthermore, sequences are not part of the 1.29 release so I was keeping the options open. Altogether, there are four data types that the library currently supports: tuple: (a, b, c) The strength of tuples is that element access is a fast as you can get. The downsides are that the size must be known (since it can't be detected without variadic macros) to access it, the maximum size of a tuple is limited to around 25, and tuples are no good for anything that requires "resizing" the tuple. array: (3, (a, b, c)) The strengths of arrays is that element access is nearly (but not quite) as fast a tuples, the size is built into the structure itself (so it is not necessary for element access), and resizing is directly supported by the library with various primitives. The downsides are that arrays have the same size limitations as tuples and they must have their size specified when they are created. Ultimately, the difference between tuples and arrays is that tuples require the size on access while arrays require the size on construction. Either way, conversion back and forth is fairly trivial. list: (a, (b, (c, BOOST_PP_NIL))) Lists are typical, well-understood, singly-linked lists. They directly support an empty state and can be very good for "algorithmic" like manipulation. As with a runtime list, random access is terrible relative to the other structures, but random access is not the typical use of lists. They are good for folding (a.k.a. accumulation) and other pursuits that often need to deal with a nil state. sequence: (a)(b)(c) // also known as "seq" by me ;) This is kind of a universal type. It is good for everything that lists are good for--except the nil state issue--and is still very fast for random access so it can replace tuples and arrays in many situations. The size is unnecessary (it can be computed easily and efficiently) for access or for construction. This structure is a great "general purpose" type. Also, you can't beat it for appending efficiency since to append seq1 to seq2 requires only: seq1 seq2.

I must admit that I haven't read the docs of Boost.PP since before 1.28 was released, so I know very little about file iteration. My apologies if these questions are already answered in the docs.

The docs (and nearly the entire library) have been rewritten with the release of 1.29--it is mostly backward compatible but not quite (there are docs that discuss incompatibilities). I highly urge people to get the latest CVS sources of the PP lib (and the PP lib docs) because it is better. In particular, I added the sequence support and full-fledge support for the array types mentioned above. As for file-iteration, it is conceptually very simple, but it requires a slightly different thinking about how preprocessing works. I personally like to think of it as an "execution path" with file iteration representing a for-loop that iterates over files (or parts of files). It is a very powerful tool--just ask the Python guys or the MPL guy (Aleksey). In any case, there is a topic devoted to file-iteration in the docs, so it might interest you to read that. Basically, the differences between 1.28 and 1.29+ versions of the library are phenomenal. There are *massive* speed increases on EDG-based preprocessors as well as significantly more functionality. Specifically, preprocessor metaprogramming has moved out of "just macros" into other interesting areas (file-iteration is an example of this).

Regards, -Tom Wenisch Computer Architecture Lab Carnegie Mellon University

If anyone here has any questions about the PP lib, I'll answer as best as I can. Regards, Paul Mensonides