Generalised reflection (was: Re: GUI Library Proposal for a Proposal)
Message: 15 Date: Wed, 10 Nov 2004 20:52:24 -0500 From: David Abrahams <dave@boost-consulting.com> Subject: [boost] Re: GUI Library Proposal for a Proposal To: boost@lists.boost.org Message-ID: <uwtwtnodj.fsf@boost-consulting.com> Content-Type: text/plain; charset=us-ascii
"George van den Driessche" <grebe@bigfoot.com> writes:
I'm stumped by how to store this information in any sort of container though, since the properties are of different types. Any thoughts, or is this madness?
The properties are of different types, but the type information for properties will always be of the same type :) So you can generate a property set for your class, that includes the dictionary that maps names to property_type_info structures. Each property_type_info would be automatically generated from the static type of the property.
The thing that's bugging me is that all the reflection mechanism is covered by Boost.Python because it relies on the same principles. I *think* the boost-langbinding list seems to know this, since you also need the reflection capabilities in order to support languages that aren't Python. But that list doesn't seem to have much traffic so I haven't much to go on.
We did some work on that project recently and presented it at the Boost workshop at OOPSLA. See http://boost-consulting.com/writing/oopsla04.html http://boost-consulting.com/writing/langbinding.ppt
In order to avoid too many reinvented wheels, I'd like to see Boost.Python split neatly into two stages:
(a) Reflect C++ types into C++ objects. Each type for which you provide a reflection map will correspond to exactly one property_set object, which would contain a set of property_type_info objects. (b) Expose the types to Python using the objects created by (a).
We do have a front-end/back-end architecture, but I'm not sure whether we're doing what you want.
Once you've made that split, then you can add: (c) Expose the types to Lua using the objects created by (a). (d) Expose the types to <language-of-your-choice> using the objects created by (a). (e) Serialise the types, using results of (a).
This I have serious doubts about. What you'd expose for Python/Lua binding would be an object's public interface: its abstraction. In order to do serialization in that way, you'd likely have to expose an object's guts: its implementation details.
Yes, but in Boost.Python you're really saying two things for each type: (a) Here's my type's internal structure. (b) Here's how to present the internal structure to Python. I think this fact is slightly disguised because the two steps are both performed by the def() function. But the fact that it takes pointers-to-members is a giveaway. Part (b) here corresponds to the extra data that you attach to exposed members, such as return_internal_reference. In a serialisation library, you're saying two things too: (a) Here's my type's internal structure. (b) Here's how to write the structure to a file, and read it back. In fact, if your classes are appropriately exposed to Python, you could just pickle them in Python and you'd be done. But as was pointed out a couple of days ago (by Beman I think), not everyone wants to use Python in their project. Maybe it would help to describe how I arrived at the idea I'm suggesting. I wanted to construct a serialisation library in C++, that would be used from C++. I quickly realised I needed to separate the description of which elements of an object should be serialised, from the description of how the serialisation should happen. That way, a single class description could be processed in many different ways, so I could use it to write XML, or plain debug text, or a compact binary version, without changing the class definition. Once I'd set about it, I realised I was duplicating the structure that I saw in Boost.Python, but instead of attaching to each property a description of how to interface with Python, I was attaching the description of how to serialise that property. And I thought, I wish I could just use class_ instances to describe the data to serialise. But of course, Boost.Python class_ instances are designed to be used from Python, not from C++ :) Obviously, this train of thought is very similar to that which gave rise to Boost.Langbinding. And looking at the usage examples for John Torjo's Win32GUI framework, I saw the same pattern arising yet again: lists of properties (that is, pairs of member functions), with attached data (in this case, about bindings to specifc UI elements, input validation routines, and so on). What I'm suggesting amounts to factoring out the common first step. Let's suppose that we do in fact have a general reflection library, which allows you to list the properties of your class and attach arbitrary bits of data to them. Then the data that you attach to a property can be: - things from Boost.Langbinding about how to get at it from script (or simply the fact that script shouldn't be able to get at it) - things about how to serialise that property (or simply the fact that it shouldn't be serialised) - hints about how to present the property in a UI (e.g. if you attached some object that says "this integer's range shouldn't exceed [0,60)" then a slider or number control could pick that up and limit itself to that range). - probably some other things I haven't thought of.
(f) Bind types to GUI objects, using results of (a).
What does it mean to bind a type to a GUI object?
(g) ... you get the point.
I'm not sure.
A couple of other people have tried to explain what I meant, and I hope the above example makes it clear. The major architectural problem I foresee with hanging different categories of information (scripting, serialisation, UI &c) off the same reflection objects is that people might want to define how to serialise an object in one place, but how to script it in another. This could arise, for example, if someone supplies a library that only includes scripting information for its types, but the library's user also wants to add serialisation support. I think you encountered a related problem in the Boost.Langbinding development where two libraries that each supply bindings for one type could conflict if both used in the same executable. Possibly it's too ambitious to try to bring all these related, but different, activities under the same umbrella. Maybe multiple property maps really are the best solution. But it's certainly worth considering, because even in that case I suspect they could share a common set of base templates. For instance, if you attach range information to a scalar property, that could automatically be used for validation by serialisation code, UI code, and scripting code. George
"George van den Driessche" <grebe@bigfoot.com> writes:
Once you've made that split, then you can add: (c) Expose the types to Lua using the objects created by (a). (d) Expose the types to <language-of-your-choice> using the objects created by (a). (e) Serialise the types, using results of (a).
This I have serious doubts about. What you'd expose for Python/Lua binding would be an object's public interface: its abstraction. In order to do serialization in that way, you'd likely have to expose an object's guts: its implementation details.
Yes, but in Boost.Python you're really saying two things for each type: (a) Here's my type's internal structure.
Emphatically no. In Boost.Python you describe a new Python type's public interface. Its interface doesn't have to have any relationship to that of the C++ type.
(b) Here's how to present the internal structure to Python. I think this fact is slightly disguised because the two steps are both performed by the def() function.
Believe me, I understand that there are two things happening there, and the Langbinding project aims to separate those two things. But when doing language binding you are definitely *not* describing internal structure. The two things are: i. Describe the external interface to the new type. ii. Turn that interface description into an actual Python type.
But the fact that it takes pointers-to-members is a giveaway.
The fact that you are generally binding member functions (and only public ones) is also a dead giveaway. What Rene said may be true (that most types can be serialized and reconstructed through their public interface), but it's not a foregone conclusion.
Part (b) here corresponds to the extra data that you attach to exposed members, such as return_internal_reference. In a serialisation library, you're saying two things too: (a) Here's my type's internal structure. (b) Here's how to write the structure to a file, and read it back. In fact, if your classes are appropriately exposed to Python, you could just pickle them in Python and you'd be done.
*If* they expose enough information to be picklable. >>> And note that Boost.Python has explicit support for pickling, which rather strongly hints that the information you supply it for ordinary class wrapping is usually not sufficient for serialization. <<<
Maybe it would help to describe how I arrived at the idea I'm suggesting.
No offense, but probably not; you're not the first one to think that the ideas behind Boost.Python could be used to build a generalized reflection library. If you search this mailing list's history and that of the Python C++-sig I think you'll find some other conversations along these lines. It's a fine idea; I just have a hard time believing that the information you need to expose for serialization is normally the same as what you need to expose for language binding. Maybe the same front-end interface code can be used to do the exposing, but whatever the user writes for one is not going to magically work for the other. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
participants (4)
-
David Abrahams -
George van den Driessche