Should nodes documentation be generated from code?

[alexisrolland]

It's awesome to have such detailed documentation about all the core nodes, but it's going to be very tedious and time consuming to maintain it in all languages. I wouldn't be surprised if part of the documentation becomes obsolete over time.

It might be optimal to:

• Maintain only English documentation.
• Generate it from the node code. The node code already has some properties that can be used suched as DESCRIPTION and input type's tooltip.

I think a fantastic reference of repository that generates documentation from code is FastAPI. It generates an entire swagger based on endpoints code.

[benceruleanlu]

hello! really appreciate the feedback🌛

very tedious and time consuming to maintain it in all languages

it may truly be!

although, I'm pretty sure that usually only the English and/or Chinese locales are hand/well-written, the rest are usually machine translated off of that.

also, there are plans to sync the embedded docs and our existing docs website, so this can be as painless as possible for technical writers.

we should probably clarify this down in our README, I'll open another issue for this

Generate it from the node code. The node code already has some properties that can be used suched as DESCRIPTION and input type's tooltip.

this is existing behaviour if comfy can't find a custom markdown file

feel free to suggest ways to extend this functionality though, it does seem somewhat bare

please let me know if there's any other pieces of feedback/suggestions/questions!

[comfyui-wiki]

I found that the docs generated from code usually lack details and need to have additional examples or tutorials added to them. But it would be great if some basic node docs could be generated by code (Currently, there are still a lot of nodes that don't have docs), or there could be a reminder to notify the maintainer that some docs are obsolete, or there could be a feedback button that allows users to directly submit a pull request (PR) or edit the docs like it in docs.comfy.org

If a node document is generated by AI from code, I think it's better to have a flag or something for the maintainer or users to know that it can be improved.

Currently, this is how I maintaining the node docs:

Usually, basic docs will be generated from code with AI, and then I need to test them and add examples, such as including images or linking to the tutorials on docs.comfy.org. I need to check the localization one by one to ensure that the node's parameter naming can match the current ComfyUI's localization. One example is this PR: #13

If the node docs in other languages are all generated by translation, sometimes the node parameter's naming won't match those in the current ComfyUI's localization (which is currently all in English), and we still need to check them manually. But I don't mind if basic node docs are generated by AI. But we still need to check them manually.

[benceruleanlu]

we can have a small CTA in the help page for core nodes to send people over to open an issue or open the web editor for that file in comfy-org/embedded-docs

feel free to open that issue in the frontend repo

[alexisrolland]

Actually when I mentioned generating nodes doc from code, I did NOT mean with AI. I though it might be feasible to do it WITHOUT AI, based on the node properties defined in code. It would be more robust, predictable and controlable than with AI but certainly requires some work.

Taking this dummy custom node as an example (based on the Nodes v2 specification):

class customNodeExample:
    """
    Some docstring to describe what the node is doing...
    This could actually be in markdown directly.
    """

    @classmethod
    def INPUT_TYPES(s):
        return {
            "required": {
                "int_field": ("INT", {
                    "default": 0, 
                    "min": 0, #Minimum value
                    "max": 4096, #Maximum value
                    "step": 64, #Slider's step
                    "tooltip": "This is an example of integer input", #Input description in the documentation
                }),
                "float_field": ("FLOAT", {
                    "default": 1.0,
                    "min": 0.0,
                    "max": 10.0,
                    "step": 0.01,
                    "round": 0.001, #The value representing the precision to round to, will be set to the step value by default. Can be set to False to disable rounding.
                    "tooltip": "This is an example of float input" #Input description in the documentation
                }),
            },
        }

    RETURN_TYPES = ("IMAGE",)
    RETURN_NAMES = ("image_output_name",)
    FUNCTION = "test"
    OUTPUT_NODE = False
    CATEGORY = "Example"

[...]

(1) The inputs' name, data type and tooltip are strictly defined in the code (although tooltip is optional). It could be possible to parse the INPUT_TYPES dictionary and generate a markdown table accordingly.

(2) Outputs are a little more tricky considering their current specification is limited. But maybe this could be improve in Nodes v3 specification so that ouputs also include descriptions / tooltips.

(3) A more verbose description of the node can be stored and parsed either from the class docstring, or from the DESCRIPTION property.

It's probably not a silver bullet to guarantee the quality of the documentation, because code owners might not provide tooltips / descriptions (or might do it in another language than English). But maybe it could serve as a fallback mechanism if there's no manual documentation.

If you think that's worth trying I could try give it a shot... Actually it might be better to do a proof of concept based on Nodes v3 specification, to be future proof... even if this still subject to changes.

[benceruleanlu]

sorry, I might be misunderstanding something, but it sounds like the feature you're requesting already exists.

if we can't find custom markdown for a node, it falls back to showing the node's description, i/o name, i/o type, and i/o tooltip, without AI.

please let me know if I'm missing something

[alexisrolland]

Hahaha ok, this is what I was thinking about... I just didn't know it exists already ;)

I think we can close this issue.