Blender 3D: Blending Into Python/Custom datablock properties

Adding support for Per Datablock Properties Proposal

A Draft, proposing the addition of user defined, per datablock properties.
This means that any blender datatype with an ID should be able to have properties assigned to it.
Object/Mesh/Camera/Metaballs/Lamp/Lattice/Text/IPO/Screen/Scene/World/Wave - This does not include faces or verts for the moment. This may be added later but would require further work and is outside the scope of this proposal.

There are some areas I do not know enough about and request others to expand.

I encourage others who may benefit from this, to read through this page and see if there are any improvements that could be made (edit, give examples and discuss :) ) - Ideasman

Discussion

Message from Ton

About the property system, what needs to be sorted out is;

- how it influences the UI
- how does hierarchical work precisely? (also for UI access)
- is there a way to default to property sets? Means;
you can set custom properties to be added by default for yafray, crystalspace, etc.
Would save a lot of hassle, and would create good accessible UIs that way.

- I'm busy and haven't time for all these now, - Ideasman

Reed Hedges says:

In response to Ton's questions above, here's what I think: In the UI, you need to add something in some pane that's exactly like the existing gamelogic properties (name, value, type {string,bool,int,float,vector,etc.}). This could be a new window type instead of the game logic window or it could be in the buttons window with the Object-level operations, or it could be part of the Outliner view. From what I read below, it looks like the hierarchical naming is primarily a naming convention. This could be used by the UI to display as a tree, or they could just be a standard list with the "hierarchies" just in the name format. To me, default properties are not useful. However, they could be a separate mode of the same gui used for individual objects.

+-CUSTOM PROPERTIES -------------------------------------+ 
|                                                        |
| Selected Object: ME:Suzanne01               (Defauts)  |  <-- (Defaults) is a two-state toggle button
|                                                        |      to show default props instead of sel. object
| Name:               Type:    Value:               Del: |   
|  [Example.Prop....] (String) [This is my stri...] (X)  |  <-- Name is text field. Type is multivalue
|  [Another.example.] (Bool  ) (true)               (X)  |      dropdown. Value depends on type.
| (New)                                                  |  <-- (New) creates a new empty property above it.
+--------------------------------------------------------+

I would also request that all object types be able to have properties, including the Scene/World. (Yep, Scene & World will since they have ID's - Ideasman42)

Applications

Annotations Comments about a mesh, object or material could be stored as a link to a text block or a string. Other metadata might also be added for different projects.
Render Settings Extra Properties for external rendering engines, may of these could be added to the material data, and global render settings could be assigned to the Scene datablock. This could be a step towards better renderman intergration!
Game Engine Many game engine specific settings such as Level Of Detail for objects. Different pose states for figures. could be stored in these settings.
Blender CAD Many attributes could be added for use in the BlenderCAD project. (please expand)
Simulation There is a gap to be filled in the simulation industry, there are a number of people already using blender in simulation, however there is no easy way to add settings for Simulation output (Namely OpenFlight Properties)
Extended datas some internal functionalities may need to store values, while not wanting to impose the added weight to all datablocks, even those who don't need it
- Comment: Allow users to mess with data that's accessed by internal functions as well is not a good idea. If it's useful for internals at all (I can't think of a situation where I wouldn't prefer another implementation) user data and Blender data should be separated or access restricted (read-only for Blender stuff). -- tbleicher
- Answer to comment : it is not because you use properties mechanism that it must be user editable or even viewable. that case is useful when you need broad datas added in very few cases like a tiny percent of edges on a whole mesh
- (by Ideasman) Guys, Id like to resolve this, feel free to disagree, but a line has to be drawn as to the functionality this project covers.
  - 1) Users must be able to edit properties directly, there will be an interface for this. Blenders internal tools can manage properties also but there will be a panel just for datablock properties.
    You could argue that letting the user touch them introduces possible errors in the data, but the advantages of allowing the user edit the properties far outweigh the effort of making python-scripts/blender correctly handle a variety of data.
  - 2)Properties will not be assigned to edges, bezier points, faces, indervidual bones etc. This is out of the scope of this project. If there is a need for pre mesh face properties for e.g., we can address this as a separate addition. for now only data with an ID struct will have custom properties.
Blender BVH Motion Capture Joint names could be assigned as properties so that duplicate object names would not conflict between different motion capture rigs. At the moment I am making all joint names unique because there is no good way to set the objects goint name. (Ideasman)

Defaults

For default settings a text file could be used in the home directory called .Bprops
Existing files files are .B.blend for the default new file and .Bfs for a list of default dirs in the fileselector.

The .Bprops text file would store properties that would be shown by default (without manually adding to the datablock)
DataBlockType:PropertyType:PropertyName=DefaultValue - Would be the following format.

Object:bool:CrystalSpace.staticObject=1
Material:Text:RenderMan.Shader=""
Scene:int:RenderMan.Quality=5
Lamp:bool:OpenFlight.LightString=0

NOTE from the perspective of b2cs (crystalspace exporter), not only defaults would be useful, but also minimum, maximum and possibly even a description for each property.

Example Usage

Here are projects that may directly benefit from properties, appretiate input from authors and expansion on how this might benefit their project. Does this proposal furfill your needs?

Blight (openflight compatibility- storing openflight spesific notes)
Make Human(make human, per mesh humen settings...)
Brad Blender/Radiance Exporter (material settings...)
- material: save description as text string; save dependencies on other definitions ("modifiers")
- mesh: use props for "special" export objects instead of meshobj names
- lamps: strore reference to luminaire data and used data source (db, fs)
- scene: render options and post process information (filters, image conversion)
CrystalSpace OpenSource Game engine.
RenderMan Adding RenderMan support to Blender.
- shader: specify shader name and parameters for RenderMan shaders (surface, displacement...)
- primitives: provide hints to RenderMan exporter for processing specific primitives.
- scene: provide hints to RenderMan exporter for scene level rendering control.

Add your project here

Property Structure

Properties will be stored in a hierarchy. each datablock can any number of property folders
(I'm proposing no root level properties to enforce better design)
A plugin that used properties would have a folder in the root for all its settings- This would stop it from getting messy in the UI, and keep namespace separate for each plugin.
Example structure. python can wrap the C data and access as a dict or as a class we may allow both to coexist ob.prop.foo or ob.prop['foo'] - Ideasman42),

object.prop.metadata.annotation = 'This is a building' # string
object.prop.metadata.timestamp = sys.time() # float
object.prop.crystalSpace.lod = Mathutils.Vector(0,50) # vec
object.prop.crystalSpace.shadowMap.used = True # bool
object.prop.crystalSpace.shadowMap.size = 512 # int

Property Types

Here are a list of property types that could be used by plugins.

bool
Int
Float
String May be limited in size, for large multiline texts a text block could be linked.
Vector (2D, 3D, 4D)
Datablock link Would allow you to link to another datablock- text/object/mesh/curve etc.
Arrays of all previous values (not a requirement, since a folder of properties could be used, could be useful)

User Interface

The OOps view can be extended to have a panel that would allow editing of each datablocks properties. This could also be viewable in the outliner, however OOps view is better suited because it shows each datablock only once. and shows all datablocks at the same time.

Python API Interface

Comment 1: For scripting more important than accessing and editing the whole set of properties will be the access to individual props by name/hierarchy. For BPy I'd like to have an implementation of __getitem__/__setitem__ (like in dictionaries). Examples:
- amesh.prop["Radiance"]["export"]["smooth"] = True # set the property, bool
- var= amesh.prop["Radiance"]["export"]["smooth"] # get the property
- amesh.prop["Radiance"]["export"].keys() # get a list of sub properties.
  ["smooth", "emit_light"]
- del amesh.prop["Radiance"]["export"]["smooth"]

KeyError should be raised whenever a non-existent key is specified (or an existing key is converted to "container"). How to check this and react is the job of the script.

Comment 2: To import/export the property set a universal way to serialize the data would be wellcome. Currently we have to use the Python standard library to store/read runtime data from the Registry/text objects/text files. A builtin way to create/parse text strings from/to Python objects could help script writers to load and save persistent data. That way the whole properties set could be stored as a string attribute of the object.

-- tbleicher

(comment: Python's pickle module provides this, though I don't know how this might interact with Blender objects, etc. -- sapir)

Internal Implementation

Any Blender Data with an ID would be able to have a property. (please expand and add to how this may be implemented - in Blenders DNA)

User data already sort of exists for objects?

Sorry, I just came upon this page searching the web trying to figure out how to add user data to objects in Blender. Couldn't find anything in my searches, but finally figured out some stuff from the API docs.

Try these in Blender's Python interactive console:
cube = bpy.data.objects["Cube"]
cube.addProperty("myUserDataName", 1, 'INT')
dir(cube.getProperty("myUserDataName"))
var = cube.getProperty("myUserDataName").data
cube.getProperty("myUserDataName").setData(2)
cube.getAllProperties()
cube.removeProperty("myUserDataName")