Page MenuHomePhabricator

db: introduce [de]serializing _DB as a file
ClosedPublic

Authored by akanad on Aug 30 2018, 2:32 AM.

Details

Reviewers
JackDanielZ
Summary

it's necessary to save _DB data as a file and load it later.
It will provide benefits such as saving time to build _DB,

easy to share a kind of a preset _DB, etc.

this patch introduce de/serializing _DB as a file.

Diff Detail

Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 7982
Build 7420: arc lint + arc unit
akanad requested review of this revision.Aug 30 2018, 2:32 AM
akanad created this revision.
akanad updated this revision to Diff 16626.Aug 30 2018, 2:37 AM

Add a name for another field.

Do we really need the DB to be stored in EET?

IMHO, Yes.

I think that those features(saving _DB & loading eet) make ea_gen more useful.
for example, we are able to make a new db set easily by mixing existing db sets.

I estimated that it would take affordable time to make it.
and that's the reason why I'm trying to write a patch for eet description of _DB and _DB related data structures since yesterday.
I will post it to here,phabricator, soon.

anyway I'd like to know your opinion.
does it take unreasonable time to implement those features?

akanad updated this revision to Diff 16634.Aug 30 2018, 7:55 PM
  • DB: Save db as eet file

It has much more complicated than I expected : )
though I couldn't finished things such as union field, I'd like to share it first to hear your opinion.

Q1.
I guess that eet could not handle once descriptors have circular dependencies. is that right?
would be there any solution or hacky one?

Q2.
there are something that I commented out like Eolian_Klass*
as far as I feel, I need to define descriptor for Eolian_Klass and all of the children of the structure to make eet working properly without modifying current code.
I haven't checked those things however, is there any other solution ? something like dropping actual data when saving, then re-assigning those data while loading eet file. I don't know..

Hi Wonki,

To support EET, you will have to modify the structure members. For example, Eolian_Class/Function should become class/function name. Concerning Enum fields, I think a string should be added. After the EET is loaded, you need to use these strings to find out the Eolian data. The stringshare cannot be use anymore.

Imo, the circular dependencies should be avoided. If you refer to obj_value->widget, use the class name there and then update the widget field after the EET loading.

I don't have any problems with that feature, as long as you do it :P

Daniel

akanad updated this revision to Diff 16927.Sep 21 2018, 4:06 AM
  • DB: Save db as eet file

    rebase
akanad updated this revision to Diff 16928.Sep 21 2018, 4:09 AM

squash two patches by a patch.

dependency descriptions

Hello @JackDanielZ
would you please help me to implement saving feature of _DB ?
It's much more complicated than I expected to save "Eolian_Data_Structures" as a differenct form then restore while load eet file.

akanad retitled this revision from Add a name onto anonymous union field to Introduce [de]serializing _DB as a file.Oct 24 2018, 6:00 PM
akanad edited the summary of this revision. (Show Details)

Hi,

I added comments on how I think EET can be used for the DB.

Everytime you use a name to store Eolian information, you should walk over the structures to determine the Eolian data pointers. Don't forget you may need to parse the Eo files if not loaded yet.

I will make my best to help you on this adventure. I have a lot of work related to Exactness tests.

Daniel

src/bin/db.c
382

Use some char *w_name_eet that will help to fill the w member.

386

Same here

407

We can remove the union as this structure is used a little and there is only two members in the union. As you wish

419

Need to EET the mode?

422

This is the hardest part:

  • eo_param: maybe add char *eo_param_eet that corresponds to the param name. It think it is sufficient to determine the eo_param during loading
  • ea_type: trivial
  • basic_type: an enum may be created instead of its type
  • union:
    • lists: trivial
    • array_string: it is supposed to be a _Parameter. Does EET support that? I dont remember.
425

Use name additionally

435

Use name additionally

438

I dont think you need to EET the hashes and the lists, as it can be filled after loading by walking over all the wdgs...

akanad added inline comments.Nov 1 2018, 6:52 PM
src/bin/db.c
386

you mean something like char *wdg_name_eet, right?

407

you know that removing union is not a big deal. however, Eolian_Enum_Type_Field* is the deal.

akanad updated this revision to Diff 17221.Nov 1 2018, 11:55 PM

working in progress

JackDanielZ added inline comments.Nov 2 2018, 3:31 AM
src/bin/db.c
119

IIrc you need char *, without const. This is the what EET gives.

167

This corresponds to the field?

Why enum is harder than class, function...?

205

I dont think you need all these types. Most are useless

akanad updated this revision to Diff 17290.Nov 8 2018, 11:41 PM
akanad retitled this revision from Introduce [de]serializing _DB as a file to db: introduce [de]serializing _DB as a file.
akanad edited the summary of this revision. (Show Details)

I added hints for some members which are needed to be restored in run-time.
And I removed a union field to reduce implementation complexity while gaining space complexity.

It looks working roughly however. it doesn't work properly,

because It has a un-restorable data such as Eolian_Function_Parameter.

Would you please give a advice to get this throught?

Hi,

I will try to work on it this week and update you asap.

Daniel

Hi,

I tried to make it work. It seems ok now, at least the DB display output is the same before and after load. I attached the patch here. Please update latest ea_gen and apply the patch.

I haven't play with the union yet, as it doesn't seem so important for the demo. I have let in the main.c some code that saves the DB output in toto1.txt (before load) and toto2.txt (after load) files. It should be removed before pushing the patch to master.

Please tell me if you have issues to make it work :-)

JackDanielZ accepted this revision.Dec 13 2018, 11:58 PM
This revision is now accepted and ready to land.Dec 13 2018, 11:58 PM
JackDanielZ closed this revision.Dec 14 2018, 12:08 AM