Re-Purpose Miss_hit Help

[ThePerfectComputer]

Making this question public here so that more people can see it:

====

Thank you for work excellent work on miss_hit. Although this isn't an externally exposed functionality, I'm wondering if I can use the miss_hit code to build something that takes a matlab source file and spits out an abstract syntax tree.

I imagine there’s already a function in the code that does exactly that?

[ThePerfectComputer]

Also, I'm the mh_debug_parser works well. The code is easy enough to read. I'm wondering if the Abstract Syntax Tree captures comments?

[florianschanda]

It does! Specifically for your problem you might also like to know that the docstrings are captured specifically:

https://github.com/florianschanda/miss_hit/blob/f01fbf9cd43c52fdb8fe6dce162c9772637ec199/miss_hit_core/m_ast.py#L329C7-L329C7

To get this to work you need to borrow the post-processing code from mh_style:

miss_hit/miss_hit_core/m_parse_utils.py

Line 31 in f01fbf9

def parse_docstrings(mh, cfg, parse_tree, tbuf):

You will also need to use the special lexer that preserves all tokens in a list, instead of throwing them away after generation.

miss_hit/miss_hit_core/mh_style.py

Line 1068 in f01fbf9

lexer = MATLAB_Lexer(wp.cfg.language,

miss_hit/miss_hit_core/mh_style.py

Line 1123 in f01fbf9

tbuf = Token_Buffer(lexer, wp.cfg)

[ThePerfectComputer]

I will note I'm seeing support for comments as tokens.

[florianschanda]

Yes, the lexer returns all comments and the parser then throws them away.

In order to associate comments (specifically the docstrings) to functions, classes, or files you need to do some post-processing similar to what mh_style or mh_copyright does:

miss_hit/miss_hit/mh_copyright.py

Lines 152 to 159 in f01fbf9

	try:
	tbuf = Token_Buffer(lexer, wp.cfg)
	parser = MATLAB_Parser(wp.mh, tbuf, wp.cfg)
	parse_tree = parser.parse_file()
	parse_docstrings(wp.mh, wp.cfg, parse_tree, tbuf)
	except Error: # pragma: no cover
	return MH_Copyright_Result(wp, False)

[ThePerfectComputer]

How can I make a modification to the matlab AST, and then write it back out to a file?

[florianschanda]

This cannot be done, mainly because e.g. comments are NOT part of the AST.
What mh_style does is a bit more subtle, we remember the original token stream (including comments) and then dump it back, modified in small ways.

• miss_hit/miss_hit_core/m_lexer.py

  Line 1327 in f01fbf9

  def replay(self):

  (where we dump it back)

• miss_hit/miss_hit_core/m_ast.py

  Line 42 in f01fbf9

  class Autofix_Instruction:

  (the datastructure used to modify the replay behaviour

If you want to add e.g. comments then you would need to extend the autofix thing to say "add a comment after this line" or similar and modify the replay function to do that.

[ThePerfectComputer]

I have some some identifier nodes and am wondering what is a good way to determine the line-col location of the indentifier node?

[ThePerfectComputer]

Right now I'm able to call .loc() on an identifier node, but I'm currently only seeing the line number and no column information.

[ThePerfectComputer]

nvm - the following work:

• node.loc().line
• node.loc().start_col
• node.loc().end_col

[florianschanda]

Glad you worked it out :)

[ThePerfectComputer]

And on to the next question - heh...

If I install miss_hit via pip3 install miss-hit-core, how do I then do something like from miss_hit_core.config import Config for example?

[florianschanda]

I'm not sure I understand that question. That should work as far as I know?

[ThePerfectComputer]

It works now. Turns out there was an issue with my pip configuration.