Generator.OutputMode.FilePerModule mode not working?

[HenrikAkseliValve]

The current version of CppSharp generates only one file with Generator.OutputMode.FilePerModule mode if header files are in the same include directory. ILibrary Setup method:

        /// Setup the driver options here.
        public void Setup(Driver driver)
        {
            //Set up the driver options.
            driver.Options.GeneratorKind = GeneratorKind.CSharp;
            driver.Options.OutputDir = "<Output_folder>/QtShatpNi/QtBase";
            driver.Options.GenerationOutputMode = GenerationOutputMode.FilePerModule;
            driver.Options.GenerateDeprecatedDeclarations = false;

            // Set up parser options.
            driver.ParserOptions.LanguageVersion = CppSharp.Parser.LanguageVersion.CPP17;
            driver.ParserOptions.IncludeDirs = [@"<QT_path>\Qt\6.7.2\msvc2019_64\include", @"<QT_path>\Qt\6.7.2\msvc2019_64\include\QtCore"];

            // Set up modules.
            //AddModule(driver, "QTypeInfo", "qtypeinfo.h");
            AddModule(driver, "QChar", "qchar.h");
            AddModule(driver, "QNamespace", "qnamespace.h");
        }

        /// <summary>
        /// Helper function to generate a module.
        /// </summary>
        /// <param name="driver"></param>
        /// <param name="modulename"></param>
        /// <param name="header"></param>
        private void AddModule(Driver driver, string modulename, string header) {
            var module = driver.Options.AddModule(modulename);
            module.OutputNamespace = _namespace;
            module.Headers.Add(header);
            module.LibraryName = modulename;
            module.SharedLibraryName = "Qt6Core.dll";
            module.IncludeDirs.Add(driver.ParserOptions.IncludeDirs[1]);
        }

I think this is caused by CleanUnitPass associating unit passes with the first module with the same include path.
In the CleanUnitPass module is determined by this method:

        private Module GetModule(TranslationUnit unit)
        {
            if (unit.IsSystemHeader)
                return Options.SystemModule;

            var includeDir = Path.GetDirectoryName(unit.FilePath);
            if (string.IsNullOrWhiteSpace(includeDir))
                includeDir = ".";
            includeDir = Path.GetFullPath(includeDir);

            return Options.Modules.FirstOrDefault(
                       m => m.IncludeDirs.Any(i => Path.GetFullPath(i) == includeDir)) ??
                   Options.Modules[1];
        }

Line return Options.Modules.FirstOrDefault seem to cause the issue. I would remove CleanUnitPass class and move the module unit association to be done at the parser rather than later.

[tritao]

Seems like we never considered this edge case, a PR with a fix is welcome, and preferably with a test.

Btw I see that you are trying to bind Qt. Are you aware of https://gitlab.com/ddobrev/QtSharp?

[HenrikAkseliValve]

Yes, I'm aware of it but considering the last update date, I decided only to take influence.

[tritao]

Yes, I'm aware of it but considering the last update date, I decided only to take influence.

Unfortunately Dimitar passed away before fully completing the work on QtSharp, but a lot of it should still be applicable.

[HenrikAkseliValve]

I have been working on this.

The trouble I have right now is that current unit tests can't find C++ test header files. They should be under a subdirectory of the Executing Assembly's folder. For my case they should be under "bin/Release_x64" but they aren't.

Project files don't copy them to the bin folder because the project does not find them.

What is your setup @tritao? Did I forget to do something?

Test header files are in the repo but not in the folder the project tries to look for.

[tritao]

If you check the GetTestsDirectory static inside GeneratorTests.cs, it tries to find the tests/dotnet by walking up through the directory tree.

We prefered that approach to copying the files since it should lead to less duplication and potential troubles.

I wonder why it's not picking it up on your setup though, do you have a custom output folder setup that is outside the main CppSharp checkout?

[HenrikAkseliValve]

The problem was that I had a test folder in the release_x64 folder for some reason. Since the check only checks that folder structure exists not that files exist it selected it before going to the parent folder.

Thanks @tritao I missed the loop part of the GetTestsDirectory.

[HenrikAkseliValve]

Had some work stuff not leading me to finish this but I'm on a side quest with this right now.

One unit test gave an error which let to me having problems with GetCXXRecordDeclFromBaseType does throw an assert for this structure in "xmemory" on windows 10:

template <class _Ty1, class _Ty2, bool = is_empty_v<_Ty1> && !is_final_v<_Ty1>>
class _Compressed_pair final : private _Ty1 { // store a pair of values, deriving from empty first
public:
    _Ty2 _Myval2;

    using _Mybase = _Ty1; // for visualization

    template <class... _Other2>
    constexpr explicit _Compressed_pair(_Zero_then_variadic_args_t, _Other2&&... _Val2) noexcept(
        conjunction_v<is_nothrow_default_constructible<_Ty1>, is_nothrow_constructible<_Ty2, _Other2...>>)
        : _Ty1(), _Myval2(_STD forward<_Other2>(_Val2)...) {}

    template <class _Other1, class... _Other2>
    constexpr _Compressed_pair(_One_then_variadic_args_t, _Other1&& _Val1, _Other2&&... _Val2) noexcept(
        conjunction_v<is_nothrow_constructible<_Ty1, _Other1>, is_nothrow_constructible<_Ty2, _Other2...>>)
        : _Ty1(_STD forward<_Other1>(_Val1)), _Myval2(_STD forward<_Other2>(_Val2)...) {}

    constexpr _Ty1& _Get_first() noexcept {
        return *this;
    }

    constexpr const _Ty1& _Get_first() const noexcept {
        return *this;
    }
};

Haven't tested this on seperated file yet. Is the problem with parent of this class coming from the template?
I did change GetCXXRecordDeclFromBaseType to print which line it encountered the problem, but I don't think that caused the issue.

[tritao]

Had some work stuff not leading me to finish this but I'm on a side quest with this right now.

One unit test gave an error which let to me having problems with GetCXXRecordDeclFromBaseType does throw an assert for this structure in "xmemory" on windows 10:

template <class _Ty1, class _Ty2, bool = is_empty_v<_Ty1> && !is_final_v<_Ty1>>
class _Compressed_pair final : private _Ty1 { // store a pair of values, deriving from empty first
public:
    _Ty2 _Myval2;

    using _Mybase = _Ty1; // for visualization

    template <class... _Other2>
    constexpr explicit _Compressed_pair(_Zero_then_variadic_args_t, _Other2&&... _Val2) noexcept(
        conjunction_v<is_nothrow_default_constructible<_Ty1>, is_nothrow_constructible<_Ty2, _Other2...>>)
        : _Ty1(), _Myval2(_STD forward<_Other2>(_Val2)...) {}

    template <class _Other1, class... _Other2>
    constexpr _Compressed_pair(_One_then_variadic_args_t, _Other1&& _Val1, _Other2&&... _Val2) noexcept(
        conjunction_v<is_nothrow_constructible<_Ty1, _Other1>, is_nothrow_constructible<_Ty2, _Other2...>>)
        : _Ty1(_STD forward<_Other1>(_Val1)), _Myval2(_STD forward<_Other2>(_Val2)...) {}

    constexpr _Ty1& _Get_first() noexcept {
        return *this;
    }

    constexpr const _Ty1& _Get_first() const noexcept {
        return *this;
    }
};

Haven't tested this on seperated file yet. Is the problem with parent of this class coming from the template? I did change GetCXXRecordDeclFromBaseType to print which line it encountered the problem, but I don't think that caused the issue.

Hard to know, can you post a fully preprocessed version of that header that reproduces the issue?

[HenrikAkseliValve]

https://gist.github.com/HenrikAkseliValve/f16a3b4991c7d3dfc708dfa8407630cf

[HenrikAkseliValve]

I think it is the parent coming from the template.

I added this to NamespaceBase.h to test and it passes and complaints about xmemory one.

class Foo {
    ;
};

template <class TBase, class TVal>
class _Compressed_pair : private Foo {
public:
    TVal Myval;

    using _Mybase = Foo;

    template <class... _Other2>
    constexpr explicit _Compressed_pair(TVal _val)
        : Foo(), Myval(_val) {}

    constexpr Foo& _Get_first() noexcept {
        return *this;
    }

    constexpr const Foo& _Get_first() const noexcept {
        return *this;
    }
};

If I change Foo to TBase then it complains about the new location.

[HenrikAkseliValve]

It seems that TemplateTypeParm is not handled in the GetCxxRecordDeclFromBaseType.

[tritao]

Cool, nice find, we have an assert there but probably not being raised due to not being a debug build.

I tried debugging your header yesterday night, but unfortunately could not get it parsing, it fails due to __declspec, even though I explicitly enabled Microsoft mode.

[HenrikAkseliValve]

Currently my GetCXXRecordDeclFromBaseType looks like this.

static clang::CXXRecordDecl* GetCXXRecordDeclFromBaseType(const clang::ASTContext& context, const clang::CXXBaseSpecifier& base, const clang::QualType& Ty)
{
    using namespace clang;

    if (auto RT = Ty->getAs<clang::RecordType>())
        return dyn_cast<clang::CXXRecordDecl>(RT->getDecl());
    else if (auto TST = Ty->getAs<clang::TemplateSpecializationType>())
        return GetCXXRecordDeclFromTemplateName(TST->getTemplateName());
    else if (auto Injected = Ty->getAs<clang::InjectedClassNameType>())
        return Injected->getDecl();
    else if (auto TTP = Ty->getAs<clang::TemplateTypeParmType>())
        return TTP->getAsCXXRecordDecl();

    // Error has occured so get the file name and line number for context.
    // Build a error message.
    const SourceManager& sourcemanager = context.getSourceManager();
    auto loc = base.getBeginLoc();

    std::ostringstream oss; // TODO: Add this to every where.
    oss << "Could not get base CXX record from type. Unhandled type: " << Ty->getTypeClassName() << ". File " << sourcemanager.getFilename(loc).str() << ":" << sourcemanager.getSpellingLineNumber(loc);

    assertm(0, oss.str().c_str());

    return nullptr;
}

New handling of TemplateTypeParmType returned NULL, so I started to step through the code to see if that causes any issues. I was stepping through hit other asserts but continued step through. It seems that they are not something that should abort. I hit a NULL error regarding std module which I think I have caused and it is not related to anything happening in Parser.cpp.

I propose that the release version starts to complain more but does not abort and with debug the abort can be disabled.

I don't really want to spend time right now figuring out how some obscure C++ code can be turned into C#.