Reflection is pretty much foundation of my new engine (at the time of writing the engine is a 18kloc code-base and a third iteration/rewrite so it is still really compact). This is contrary to the past versions of the engine that had a sort of reflection that were added at later stages, were rushed and sort of incomplete - this was bad and this time I decided to take another route. The engine is still progressing and the reflection system is not yet completed so everything in this article might not reflect the actual state. I will try to blog regulary if I decide to make any crucial changes or come up with new ideas that alter how the reflection works.
One of approaches I did try was automatic registration. The idea was really simple - constructors with side effects and static instances. The idea was simple and working quite nicely till it stopped. Long story short, lessons learned and I have only one thing to say about it - you just can't trust the linker and it is really unportable and unreliable. Linker is free to strip off symbols that he sees fit (from his point of view, such that are not directly used) and this happens especially when building static and dynamic libraries. The whole point of reflection is the ability to create instances by name etc. so this was a real scenario that some types might have been only used in the executable via the reflection mechanism. Of course I am aware of the fact that you can instruct the linker to leave specific symbols - at least this can be done on Visual Studio. I am not sure if there are similar instructions for GCC and others but this was enough for me to drop this brilliant idea and move on to something more obvious, reliable and most importantly portable - simple manual registration.
Last time I posted a short snippet showing reflection code, this is a good place to start going into the details. To recap here it is:
CLASS(Component)
{
MetaBuilder::Emit<Component>()
[UIReadOnly, ScriptHidden]
->AddProperty("Id", &GetId)
->AddProperty("Hash", &GetHash)
->AddProperty("Name", &GetName, &SetName)
->AddProperty("Enabled", &IsEnabled, &SetEnabled)
->AddProperty("Dead", &IsDead, &SetDead);
[UIEntityPicker]
->AddProperty("Owner", &GetOwner, &SetOwner)
}
It is quite obvious that CLASS is a macro and it resolves to the following:
#define TW_CLASS_IMPL(clazz) const tw::Type* clazz::_Type = nullptr; \
const Type* clazz::GetType() const { return _Type; } \
const HashedString clazz::_typeName(#clazz, Djb(#clazz, sizeof(#clazz)-1));
#define CLASS(clazz) TW_CLASS_IMPL(clazz); \
void clazz::RegisterType()
It sets the static instance of variable _Type (yes I am aware of the underscore and capital letter being bad choice, I just didn't have time to refactor this tiny detail yet) to null, defines a member function that is a getter for this static instance and finally initializes another static field named _typeName (this was added later hence the almost standard conforming small letter).
The HashedString thingy is just a pair of regular c-string and hash value. The hash itself must be calculated therefore it is up to the user of the class to provide hashing function. The constructor call is pretty obvious. I use this only with const char* that are eventually embedded somewhere in the DATA section of the executable so I really don't care about copying the string. sizeof in this macro is also well defined and frees the program from having to do strlen. The hashing algorithm I use is a Djb2 function which I used a lot, never had problems with and is incredibly compact and fast.
So the macro does really simple stuff and starts the definition of yet another function called RegisterType. This is a function that each class must implement and the example implementation is seen above. This is the place where all additional meta data are being constructed and incorporated into the definition for given type.
MetaBuilder on the other hand is something that was invented later and is kind of a bastard child of my previous two approaches. But before I delve into the details of MetaBuilder and the Emit function I will present the basic two classes that hold all the meta-data the builder produces (kind of).
The host for type information is Type class. It is pretty much the thing (with some unnecessary parts hidden):
class Type
{
public:
// Default ctor for array creation
TW_INLINE Type() : m_size(0) {}
Type(const char* name, u32 hash, const size_t size);
Type(const char* name, u32 hash, const size_t size, const Class* extra);
TW_INLINE const Class* GetClassInfo() const {
return m_extraInfo;
}
TW_INLINE const u32 GetGuid() const {
return m_guid;
}
TW_INLINE const u64 GetCcid() const {
return m_ccid;
}
TW_INLINE const char* GetName() const {
return m_name;
}
TW_INLINE const u32 GetHash() const {
return m_hash;
}
TW_INLINE const u32 GetSize() const {
return m_size;
}
private:
friend class ClassBuilder;
u32 m_hash;
const char* m_name;
size_t m_size;
const Class* m_extraInfo;
u64 m_ccid;
u32 m_guid;
};
As you can see no magic in here, most crucial (and obligatory) information are held in this class - name with its hash, size, GUID and CCID I talked about last time, and a pointer of type Class to the meta-meat part. This design was the effect of various approaches I tested and it is the most flexible, yet elegant solution - at least I thought so 10 minutes ago, you might always try to change my mind in the comments ;-).
The problem I had previously is that not all types have constructors, destructors, some of them have only a base class, and some of them implement 10 other interfaces. POD's have only fields whereas regular classes contain also Properties and Methods. What is important is that I didn't want to clutter the memory with pointers, structures and other stuff that might not even be there. When I was doing some research and considering my options I stumbled upon a few implementation of reflection mechanisms, one of them had huge fat classes that contained type information. It was nothing uncommon for them to have an array of 50 characters to represent class name, 50 characters to represent method name, std::maps and std::vectors as containers for other information that could not have been there in the first place. Even empty map and vector have a cost and if you thing about it - reflection system can represent a lot of small chunks of the code-base. Properties/Functions/Enumerations/Types - if you waste some memory on each, it can be quite a footprint on the whole system. Unfortunately when you minimize the impact of bad design in one place, you probably introduce an overhead in another. Mine fault was going too crazy on templates and meta-programming and it resulted in some refactoring to meet the goals in the middle.
The first attempt I made was going nuts with design that boiled down to something like:
Abstract, Interface, Concrete classes and the derived classes that were templated - TAbstract, TInterface, TConcrete - each of these were variadic templates taking various arguments. Nice thing about those was that I could create instance of for example TInterface<ISerializable, 0, 2> clazz; that denoted interface with 0 properties and 2 methods. These arguments were passed to the templated array with size as its argument. No more data was allocated then necessary - but the obvious problem was the amount of code that needed to be generated and potentially problematic compile times in the future. The current approach is more relaxed and not so heavily templated although a lot of the code still uses templates and meta-programming. But I will come back to this at another time.
Coming back to the Type class - the static instance that each class is carrying pointer to is actually held elsewhere - (note to self: duh... seriously did I just write that?) - to be more precise each Type instance that eventually is hooked up with a pointer is managed by the Reflection System and all of those little Type structures are in contiguous memory that is preallocated. The idea was that it should increase the performance of type look-up by assuring that each entry is next to another. At the moment the structure itself is 28 bytes (as far as I remember) therefore it fits the single cache line as a whole and catches another 4 bytes of the following record which happens to be another's type hash which can be examined if the currently checked type did not pass the test for lookup by name/hash. If the next test is a missmatch I can skip to the record i+2 skipping loading next entry into the cache. This was the idea, the reality is that currently I just do a regular loop and wait for optimizing and performance tuning at later stages. Maybe in the future I will investigate an approach more close to the structure of arrays approach rather than array of structures. Of course these are tiny details that are not crucial to the performance but if something can be faster why not make it faster?
You can imagine that type information is retrieved by using the interface of the mentioned ReflectionSystem. In short it has two methods that let you retrieve Type* pointer for a type given its name or hash of the name. The first one translates to the second, just does the hashing for you. This is essentially runtime type introspection but there is also much needed compile time type retrieval and it boils down to a single templated free function: GetType. This has the following definition - and take care as I am almost certain that I had to fix something in here and it was long forgotten ;-)
// GetType For Primitives
template<typename TType>
typename std::enable_if<IsPrimitive<typename std::remove_reference<TType>::type>::value, const Type*>::type
GetType()
{
return GetPrimitiveType<TType>();
}
// For non pointer, non primitive types (?)
template<typename TType>
typename std::enable_if<!std::is_pointer<TType>::value && !IsPrimitive<TType>::value, const Type*>::type
GetType()
{
assert(g_reflectionSystem);
// Removing reference
const HashedString typeName = name_of<
typename std::remove_const<std::remove_reference<TType>::type>::type>();
return g_reflectionSystem->FindByHashedString(typeName);
}
// For non primitive pointer types
template<typename TType>
TW_INLINE typename std::enable_if<std::is_pointer<TType>::value && !IsPrimitive<TType>::value, const Type*>::type
GetType()
{
return ::details::_get_as_pointer_type<
std::remove_const<std::remove_pointer<TType>::type>::type>();
}
The middle GetType overload calls a function whose name implies searching - that is why I wanted it to be as fast as possible. GetType seems innocent and suggests fast retrieval. On the other hand thanks to such overloading the implementation for primitives is just a function call that returns a pointer - it is the fastest possible implementation. This was also essential since GetType is called very often - when deducing arguments for reflected methods, properties fields etc. and primitives are used very often.
The usage is simple as that:
const Type* type = GetType<int>();
Last thing I want to mention before finishing this up is the base class - yes, there have to be one at some point ;-) In fact this could have been optional but I decided to go with this approach for a couple of reasons. Mainly it enforces some obligatory stuff on the implementer by using pure virtual functions in the base. Also the idea was that using reflection I should be able to have automatic serialization and script binding. I just assumed that having a base class will eventually make those things easier to implement so here it is, although not so useful at the moment:
class Object : public ISerializable
{
public:
REFLECTION_DECLARE
virtual ~Object() {}
bool InheritsFrom(const TypeInfo* type) const;
virtual void Serialize(Store* store) const;
virtual void Deserialize(const Store* store);
virtual const char* ToString() const;
virtual const char* GetTypeString() const;
};
As it turns out, writing about this stuff seems even more difficult then making it ;-) hope this time there was something worth reading. I am aware that this is kind of lengthy and might lack the pearls but I will try better next time. and speaking of the devil - next time I plan to write more on how the information is being reflected and tied up together. Also show very briefly how properties, methods and fields reflection is working - this kind of stuff probably will need more detailed explanation in their own posts. Also coming up next - probably in part 4 I guess - the Annotation system some why's and how's ;)
So till next time.
No comments:
Post a Comment