Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sjad #4473

Draft
wants to merge 114 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
849de05
Add a new child action "orchestrate". And create a sample azure.yaml …
rujche Sep 6, 2024
acee60b
Get list of pom files.
rujche Sep 9, 2024
88d4d1b
add the code for azd java analyzer
saragluna Sep 10, 2024
6ba0816
Hook the java analyzer with the azd init.
saragluna Sep 11, 2024
6a4b664
Enhance
saragluna Sep 11, 2024
bafcbc4
Implement java_project_bicep_file_generator.go.
rujche Sep 11, 2024
b69a47e
Improve log: Add information about the file path.
rujche Sep 11, 2024
69bc59a
Enhance the java analyzer
saragluna Sep 13, 2024
8d67fc5
Add feature: Support add mysql when run "azd init".
rujche Sep 13, 2024
91900aa
Delete java_project_bicep_file_generator.go.
rujche Sep 13, 2024
2da438b
Add logic about accessing mysql in aca.
rujche Sep 14, 2024
f4e00ef
Merge pull request #1 from rujche/rujche/java-analyzer
saragluna Sep 14, 2024
2aa6cac
Fix typo by changing "DbMysql" to "DbMySql".
rujche Sep 14, 2024
cace7fa
Merge pull request #2 from rujche/rujche/java-analyzer
saragluna Sep 14, 2024
082b9b2
Use managed-identity instead of username and password. Now it has err…
rujche Sep 18, 2024
b0c39a6
Access MySql by managed identity instead of username&password.
rujche Sep 20, 2024
f90416f
Add Azure Deps in appdetect.go
saragluna Sep 23, 2024
84b7852
Customize the azd VS Code extension
saragluna Sep 23, 2024
6587f83
improve the java analyzer for event-driven
saragluna Sep 23, 2024
103a005
refactor the java analyzer
saragluna Sep 26, 2024
2e01347
Create service connector by bicep file.
rujche Sep 27, 2024
708681a
1. Remove duplicated 'azd extension add'.
rujche Sep 29, 2024
353a802
Update name of resources: linkerCreatorIdentity and appLinkToMySql
rujche Sep 29, 2024
6d853de
Merge remote-tracking branch 'saragluna/java-analyzer' into rujche/ja…
rujche Sep 29, 2024
85ec20b
Merge pull request #3 from rujche/rujche/java-analyzer
saragluna Sep 29, 2024
2856fb9
Merge remote-tracking branch 'saragluna/java-analyzer' into rujche/ja…
rujche Sep 29, 2024
b70878e
1. Add rule about postgresql in project_analyzer_java.go.
rujche Oct 7, 2024
437eeb6
Fix the error about CORS.
rujche Oct 7, 2024
3410c97
Merge pull request #4 from rujche/rujche/java-analyzer
saragluna Oct 8, 2024
f244293
Merge branch 'main' into saragluna/java-analyzer
rujche Oct 8, 2024
df2f5f2
add support for service bus
saragluna Oct 8, 2024
b6e6ecc
fix servicebus
saragluna Oct 8, 2024
073d857
Remove the logic of create tag after create service connector.
rujche Oct 8, 2024
ef73ce1
Merge remote-tracking branch 'saragluna/java-analyzer' into rujche/ja…
rujche Oct 8, 2024
4c84661
Merge pull request #5 from rujche/rujche/java-analyzer
rujche Oct 8, 2024
f1e2fc1
support both mi and connection string for service bus
saragluna Oct 9, 2024
23362f8
For PostgreSQL, support both password and passwordless.
rujche Oct 10, 2024
193f054
For MySQL, support both password and passwordless.
rujche Oct 11, 2024
48cbeb5
Remove logic of adding tag after creating service connector. Because …
rujche Oct 11, 2024
9f4fe27
Fix bug: create service connector only work for the first time run of…
rujche Oct 11, 2024
207ffa3
Merge pull request #6 from rujche/rujche/java-analyzer
rujche Oct 12, 2024
0594412
Add new feature: analyze project to add Mongo DB.
rujche Oct 12, 2024
c45382a
Delete unused content in main.bicept.
rujche Oct 12, 2024
dca325f
Merge pull request #7 from rujche/rujche/java-analyzer
rujche Oct 12, 2024
babf604
Fix bug: Get auth type should only be required for MySQL and PostgreSQL.
rujche Oct 14, 2024
12b169e
Merge pull request #8 from rujche/rujche/java-analyzer
rujche Oct 14, 2024
a2a3a73
Make sure app work well after deployed to ACA no matter what value "s…
rujche Oct 15, 2024
2f4d2e9
Merge pull request #9 from rujche/rujche/java-analyzer
rujche Oct 15, 2024
a83f7d7
Implement feature: detect port in Dockerfile.
rujche Oct 15, 2024
35f762c
Merge pull request #10 from rujche/rujche/java-analyzer
rujche Oct 15, 2024
de2e922
Implement feature: detect redis by analyzing pom file.
rujche Oct 15, 2024
8607043
Merge pull request #11 from rujche/rujche/java-analyzer
rujche Oct 15, 2024
a482496
Detect Mongo DB by dependency spring-boot-starter-data-mongodb-reacti…
rujche Oct 15, 2024
671902a
Merge pull request #12 from rujche/rujche/java-analyzer
rujche Oct 15, 2024
9c53e73
Support all kinds of properties file like application(-profile).yaml(…
rujche Oct 16, 2024
6e14a1b
Merge pull request #13 from rujche/rujche/java-analyzer
rujche Oct 16, 2024
6c117ee
Merge the java analyzer code from the main branch to sjad (#1)
saragluna Oct 22, 2024
ea4083f
Merge branch 'main' into feature/sjad
saragluna Oct 24, 2024
b44c961
Merge branch 'main' into feature/sjad
saragluna Oct 25, 2024
f8a10da
remove the orchestrate command (#2)
saragluna Oct 25, 2024
a6e2da5
fix ut
saragluna Oct 28, 2024
d1cd151
fix ut
saragluna Oct 29, 2024
a8f91b1
Support detecting event hubs by analyzing pom.xml and application.yml…
rujche Oct 29, 2024
d992c70
Support detect Azure Event Hubs - 2 (#6)
rujche Oct 30, 2024
ef8ebe2
switch to service bus avm (#7)
saragluna Oct 30, 2024
39fda8c
Support detect Azure Event Hubs, connect by connection string. (#8)
rujche Nov 1, 2024
2a21115
Use to AVM to support PostgreSql (#9)
rujche Nov 4, 2024
65cbfe1
Add a span to indicate if java detector has started or finished (#10)
haoozhang Nov 4, 2024
030ec5b
Use to AVM to support MySql: Auth by managed identity and username & …
rujche Nov 5, 2024
41331c0
output resources from app init (#4)
saragluna Nov 5, 2024
73810a1
Merge branch 'main' into feature/sjad
saragluna Nov 5, 2024
b5069e5
update azure yaml schema to support `resources` (#12)
haoozhang Nov 5, 2024
31f8240
Update azure.yaml schema reference for private preview (#13)
haoozhang Nov 6, 2024
08e3776
Support this scenario: frontend + backend + MongoDB (#14)
rujche Nov 11, 2024
209e859
Azd enhancement for cosmosdb (#15)
rujche Nov 11, 2024
eb356d4
Auto-detecting Redis in spring boot applications. (#16)
rujche Nov 11, 2024
0d66981
Merge branch 'main' into feature/sjad
saragluna Nov 11, 2024
9bdd8d6
fix ut
saragluna Nov 11, 2024
9ba423a
Delete service names because they are not used. (#17)
rujche Nov 12, 2024
e161608
Add support for Service Bus JMS (#19)
saragluna Nov 12, 2024
efd325d
Only provide "SPRING_DATASOURCE_URL" environment variable when using …
rujche Nov 12, 2024
5ee0aa0
Support detecting Kafka by analyzing pom.xml and application.yml (#18)
haoozhang Nov 12, 2024
18a7cbc
Change all spring related environment variable to lower-case-with-das…
rujche Nov 12, 2024
f70b046
Output the resources to azure.yaml when enable compose mode (#22)
saragluna Nov 13, 2024
d6b8040
convert resources to infraspec (#23)
saragluna Nov 13, 2024
6ed094e
Fix error when run azd init (#24)
rujche Nov 14, 2024
f05aa04
Merge branch 'main' into feature/sjad
saragluna Nov 14, 2024
2e14836
Fix error that ".DbMySql" not take effect in resources.bicept. (#25)
rujche Nov 14, 2024
3b33dc6
Fix error about marshal yaml (#26)
rujche Nov 15, 2024
a0b3d6b
Prompt when Kafka detected but no spring-cloud-azure dependency found…
haoozhang Nov 15, 2024
c04905c
fix code to support servicebus in azd composability (#29)
saragluna Nov 19, 2024
9f9c553
Handle environment variable placeholder in property file. (#31)
rujche Nov 19, 2024
521d706
Inject AzureEventHubsKafkaAutoConfiguration package path based on spr…
haoozhang Nov 19, 2024
3727ead
Fix some bindings specified by uses keywork not work (#30)
rujche Nov 19, 2024
feb1cf5
support eventhubs when using azd composability (#33)
saragluna Nov 20, 2024
c3e9caf
Add the missed codes for "storage" (#36)
rujche Nov 20, 2024
bf02b93
Add values for all environment variables (#35)
rujche Nov 20, 2024
8ca2115
Improve the prompt message for database (#37)
rujche Nov 20, 2024
27f2cc5
1. Add log about detecting rule. 2. Avoid duplicated queue name for s…
rujche Nov 21, 2024
1dc2e3e
Users can continue to use azd when java analyzer fails (#39)
rujche Nov 21, 2024
4c89041
fix code to support servicebus jms with managed identity in azd compo…
saragluna Nov 21, 2024
06ca3c5
Update azure yaml schema (#32)
haoozhang Nov 22, 2024
c8351bf
Fix code to support kafka when azd compose enabled (#42)
haoozhang Nov 22, 2024
419b699
Refactor code (#41)
rujche Nov 22, 2024
9b5d218
support cosmos when azd compose enabled (#43)
haoozhang Nov 22, 2024
ace3a60
Update yaml language server (#44)
rujche Nov 22, 2024
d8f8499
Merge branch 'main' into feature/sjad
saragluna Nov 22, 2024
7d5bdd6
Add "storageAccountResource" in azure.yaml.json (#45)
rujche Nov 22, 2024
8c72593
fix test
saragluna Nov 22, 2024
a7ef892
Fix bug: Deploy failed for the second time when resource name too lon…
rujche Nov 25, 2024
817ce35
replace placeholders when reading pom.xml (#50)
haoozhang Nov 28, 2024
40c305d
Avoid maintain env list in multiple place (#46)
rujche Nov 28, 2024
f74aa50
Update the wording or azd init in VS Code extension (#51)
saragluna Nov 28, 2024
f4eddd0
Fix pipeline failures. (#52)
rujche Nov 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions cli/azd/.vscode/cspell.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,13 @@ overrides:
- cloudapp
- mediaservices
- msecnd
- filename: internal/tracing/fields/fields.go
words:
- azuredeps
- filename: internal/appdetect/java.go
words:
- springframework
- eventhubs
- filename: docs/docgen.go
words:
- alexwolf
Expand Down
48 changes: 48 additions & 0 deletions cli/azd/internal/appdetect/appdetect.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ const (
DbPostgres DatabaseDep = "postgres"
DbMongo DatabaseDep = "mongo"
DbMySql DatabaseDep = "mysql"
DbCosmos DatabaseDep = "cosmos"
DbSqlServer DatabaseDep = "sqlserver"
DbRedis DatabaseDep = "redis"
)
Expand All @@ -122,6 +123,8 @@ func (db DatabaseDep) Display() string {
return "MongoDB"
case DbMySql:
return "MySQL"
case DbCosmos:
return "Cosmos DB"
case DbSqlServer:
return "SQL Server"
case DbRedis:
Expand All @@ -131,6 +134,48 @@ func (db DatabaseDep) Display() string {
return ""
}

//type AzureDep string

type AzureDep interface {
ResourceDisplay() string
}

type AzureDepServiceBus struct {
Queues []string
IsJms bool
}

func (a AzureDepServiceBus) ResourceDisplay() string {
return "Azure Service Bus"
}

type AzureDepEventHubs struct {
Names []string
UseKafka bool
SpringBootVersion string
}

func (a AzureDepEventHubs) ResourceDisplay() string {
return "Azure Event Hubs"
}

type AzureDepStorageAccount struct {
ContainerNames []string
}

func (a AzureDepStorageAccount) ResourceDisplay() string {
return "Azure Storage Account"
}

type SpringCloudAzureDep struct {
}

func (a SpringCloudAzureDep) ResourceDisplay() string {
return "Spring Cloud Azure Starter"
}

const UnknownSpringBootVersion string = "unknownSpringBootVersion"

type Project struct {
// The language associated with the project.
Language Language
Expand All @@ -141,6 +186,9 @@ type Project struct {
// Experimental: Database dependencies inferred through heuristics while scanning dependencies in the project.
DatabaseDeps []DatabaseDep

// Experimental: Azure dependencies inferred through heuristics while scanning dependencies in the project.
AzureDeps []AzureDep

// The path to the project directory.
Path string

Expand Down
8 changes: 8 additions & 0 deletions cli/azd/internal/appdetect/appdetect_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,10 @@ func TestDetect(t *testing.T) {
Path: "java-multimodules/application",
DetectionRule: "Inferred by presence of: pom.xml",
DatabaseDeps: []DatabaseDep{
DbMongo,
DbMySql,
DbPostgres,
DbRedis,
},
},
{
Expand Down Expand Up @@ -130,8 +132,10 @@ func TestDetect(t *testing.T) {
Path: "java-multimodules/application",
DetectionRule: "Inferred by presence of: pom.xml",
DatabaseDeps: []DatabaseDep{
DbMongo,
DbMySql,
DbPostgres,
DbRedis,
},
},
{
Expand Down Expand Up @@ -163,8 +167,10 @@ func TestDetect(t *testing.T) {
Path: "java-multimodules/application",
DetectionRule: "Inferred by presence of: pom.xml",
DatabaseDeps: []DatabaseDep{
DbMongo,
DbMySql,
DbPostgres,
DbRedis,
},
},
{
Expand Down Expand Up @@ -199,8 +205,10 @@ func TestDetect(t *testing.T) {
Path: "java-multimodules/application",
DetectionRule: "Inferred by presence of: pom.xml",
DatabaseDeps: []DatabaseDep{
DbMongo,
DbMySql,
DbPostgres,
DbRedis,
},
},
{
Expand Down
71 changes: 51 additions & 20 deletions cli/azd/internal/appdetect/java.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ import (
"context"
"encoding/xml"
"fmt"
"github.com/azure/azure-dev/cli/azd/internal/tracing"
"github.com/azure/azure-dev/cli/azd/internal/tracing/fields"
"io/fs"
"maps"
"log"
"os"
"path/filepath"
"slices"
"regexp"
"strings"
)

Expand All @@ -23,10 +25,13 @@ func (jd *javaDetector) Language() Language {
func (jd *javaDetector) DetectProject(ctx context.Context, path string, entries []fs.DirEntry) (*Project, error) {
for _, entry := range entries {
if strings.ToLower(entry.Name()) == "pom.xml" {
tracing.SetUsageAttributes(fields.AppInitJavaDetect.String("start"))
pomFile := filepath.Join(path, entry.Name())
project, err := readMavenProject(pomFile)
if err != nil {
return nil, fmt.Errorf("error reading pom.xml: %w", err)
log.Printf("Please edit azure.yaml manually to satisfy your requirement. azd can not help you "+
"to that by detect your java project because error happened when reading pom.xml: %s. ", err)
return nil, nil
}

if len(project.Modules) > 0 {
Expand All @@ -45,15 +50,18 @@ func (jd *javaDetector) DetectProject(ctx context.Context, path string, entries
}

_ = currentRoot // use currentRoot here in the analysis
result, err := detectDependencies(project, &Project{
result, err := detectDependencies(currentRoot, project, &Project{
Language: Java,
Path: path,
DetectionRule: "Inferred by presence of: pom.xml",
})
if err != nil {
return nil, fmt.Errorf("detecting dependencies: %w", err)
log.Printf("Please edit azure.yaml manually to satisfy your requirement. azd can not help you "+
"to that by detect your java project because error happened when detecting dependencies: %s", err)
return nil, nil
}

tracing.SetUsageAttributes(fields.AppInitJavaDetect.String("finish"))
return result, nil
}
}
Expand All @@ -66,6 +74,7 @@ type mavenProject struct {
XmlName xml.Name `xml:"project"`
Parent parent `xml:"parent"`
Modules []string `xml:"modules>module"` // Capture the modules
Properties Properties `xml:"properties"`
Dependencies []dependency `xml:"dependencies>dependency"`
DependencyManagement dependencyManagement `xml:"dependencyManagement"`
Build build `xml:"build"`
Expand All @@ -79,6 +88,15 @@ type parent struct {
Version string `xml:"version"`
}

type Properties struct {
Entries []Property `xml:",any"` // Capture all elements inside <properties>
}

type Property struct {
XMLName xml.Name
Value string `xml:",chardata"`
}

// Dependency represents a single Maven dependency.
type dependency struct {
GroupId string `xml:"groupId"`
Expand Down Expand Up @@ -110,8 +128,16 @@ func readMavenProject(filePath string) (*mavenProject, error) {
return nil, err
}

var initialProject mavenProject
if err := xml.Unmarshal(bytes, &initialProject); err != nil {
return nil, fmt.Errorf("parsing xml: %w", err)
}

// replace all placeholders with properties
str := replaceAllPlaceholders(initialProject, string(bytes))

var project mavenProject
if err := xml.Unmarshal(bytes, &project); err != nil {
if err := xml.Unmarshal([]byte(str), &project); err != nil {
return nil, fmt.Errorf("parsing xml: %w", err)
}

Expand All @@ -120,24 +146,29 @@ func readMavenProject(filePath string) (*mavenProject, error) {
return &project, nil
}

func detectDependencies(mavenProject *mavenProject, project *Project) (*Project, error) {
databaseDepMap := map[DatabaseDep]struct{}{}
for _, dep := range mavenProject.Dependencies {
if dep.GroupId == "com.mysql" && dep.ArtifactId == "mysql-connector-j" {
databaseDepMap[DbMySql] = struct{}{}
}
func replaceAllPlaceholders(project mavenProject, input string) string {
propsMap := parseProperties(project.Properties)

if dep.GroupId == "org.postgresql" && dep.ArtifactId == "postgresql" {
databaseDepMap[DbPostgres] = struct{}{}
re := regexp.MustCompile(`\$\{([A-Za-z0-9-_.]+)}`)
return re.ReplaceAllStringFunc(input, func(match string) string {
// Extract the key inside ${}
key := re.FindStringSubmatch(match)[1]
if value, exists := propsMap[key]; exists {
return value
}
}
return match
})
}

if len(databaseDepMap) > 0 {
project.DatabaseDeps = slices.SortedFunc(maps.Keys(databaseDepMap),
func(a, b DatabaseDep) int {
return strings.Compare(string(a), string(b))
})
func parseProperties(properties Properties) map[string]string {
result := make(map[string]string)
for _, entry := range properties.Entries {
result[entry.XMLName.Local] = entry.Value
}
return result
}

func detectDependencies(currentRoot *mavenProject, mavenProject *mavenProject, project *Project) (*Project, error) {
detectAzureDependenciesByAnalyzingSpringBootProject(currentRoot, mavenProject, project)
return project, nil
}
Loading
Loading