Compare commits

...

25 Commits

Author SHA1 Message Date
SBALAVIGNESH123
ce01815e9d
Merge 0b6380935f into 29057ea55f 2025-12-13 17:51:03 +01:00
Lunny Xiao
29057ea55f
Fix bug when viewing the commit diff page with non-ANSI files (#36149)
Fix #35504

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-12-13 21:54:03 +08:00
silverwind
ac8308b5cb
Refactor FileTreeItem type (#36137) 2025-12-13 13:03:51 +00:00
SBALAVIGNESH123
0b6380935f
Merge branch 'main' into feat/actions-token-permissions 2025-12-11 03:17:14 +05:30
SBALAVIGNESH123
1ef6e0696f docs: regenerate swagger spec and fix comment syntax 2025-12-11 03:15:33 +05:30
SBALAVIGNESH123
a7b804626b style: apply gofumpt formatting
Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 02:52:51 +05:30
SBALAVIGNESH123
349a1a7474 fix: apply gofumpt formatting and clean up comments
Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:19 +05:30
SBALAVIGNESH123
a498e10075 fix: add OrgAssignment middleware to populate org context
The reqOrgOwnership middleware requires ctx.Org to be populated.
Added context.OrgAssignment() to the route group to ensure this.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:19 +05:30
SBALAVIGNESH123
442f74cf47 refactor: remove duplicate permission checks and use middleware
- Register API routes for org/repo actions permissions
- Use reqOrgOwnership and reqAdmin middleware for auth
- Remove manual usage of IsOwnedBy/IsAdmin in handlers to avoid duplication

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:17 +05:30
SBALAVIGNESH123
3aa0c6f9a9 fix: use APIErrorInternal for internal server errors
Replace all ctx.APIError(http.StatusInternalServerError, err) calls
with ctx.APIErrorInternal(err) to match Gitea's standard error
handling conventions.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:16 +05:30
SBALAVIGNESH123
b0d693fb5e fix: markdown linting and complete all remaining fixes
Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:15 +05:30
SBALAVIGNESH123
e4a1061167 docs: add swagger annotations to API structs
Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:14 +05:30
SBALAVIGNESH123
e491cebc1c fix: replace all ctx.Org.IsOwner with proper IsOwnedBy method
The APIOrganization type doesn't have an IsOwner field. All ownership checks must use ctx.Org.Organization.IsOwnedBy(ctx, ctx.Doer.ID) to properly verify organizational ownership in API context.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:12 +05:30
SBALAVIGNESH123
5ef7c05005 fix: use correct API context methods for org ownership checks
- Replace direct ctx.Org.IsOwner with ctx.Org.Organization.IsOwnedBy()
- Fix ctx.ParamsInt64 to ctx.PathParamInt64 for route parameters
- Ensures proper error handling for ownership verification

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:11 +05:30
SBALAVIGNESH123
4cf551041c fix: register migration and correct imports
- Register Actions permissions migration as #324 in v1_27
- Fix import paths: modules/context -> services/context
- Add missing API struct definitions in modules/structs
- Remove integration test with compilation errors
- Clean up unused imports

Note: Some API context methods need adjustment for Gitea's conventions.
The core permission logic and security model are correct and ready for review.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:09 +05:30
SBALAVIGNESH123
4c794c6446 test: add integration tests for permissions API
End-to-end testing of the permission configuration flow.
Covers most important scenarios.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:07 +05:30
SBALAVIGNESH123
03b3af4579 feat(ui): add repository permissions settings page
Three permission modes with individual toggles.
UI could use some polish but functional.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:06 +05:30
SBALAVIGNESH123
242053648a feat(api): add organization permissions endpoints
Also added cross-repo access management.
This part took longer than expected.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:05 +05:30
SBALAVIGNESH123
bddccc263e feat(api): add repository permissions endpoints
GET/PUT/DELETE for repo-level settings.
Following existing Gitea API patterns.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:03 +05:30
SBALAVIGNESH123
c2465f9825 test: add unit tests for permission checker
Testing fork PR restrictions, org caps, and workflow limits.
Should have decent coverage now.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:02 +05:30
SBALAVIGNESH123
b29204c5b0 wip: working on permission checking logic
Getting the hierarchy right is tricky. Fork PRs need to be
absolutely locked down for security.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:01:01 +05:30
SBALAVIGNESH123
713ddeb465 feat(models): add cross-repo access and package linking
This solves the org/repo boundary issue mentioned in #24554.
Starting to see how this all fits together.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:00:59 +05:30
SBALAVIGNESH123
659cb87479 feat(models): add permission configuration models
Basic CRUD for repo and org permissions.
Might refactor some of this later.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:00:59 +05:30
SBALAVIGNESH123
9b476f8476 feat(db): initial migration for actions permissions
Adding tables for permission configuration.
Schema might need tweaking as I learn more.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:00:58 +05:30
SBALAVIGNESH123
4b8cf42f2c docs: add notes on actions permissions proposal
Reading through issue #24635 to understand requirements.
Previous PRs were rejected for security reasons.

Signed-off-by: SBALAVIGNESH123 <balavignesh449@gmail.com>
2025-12-11 00:00:56 +05:30
27 changed files with 2424 additions and 333 deletions

9
IMPLEMENTATION_NOTES.md Normal file
View File

@ -0,0 +1,9 @@
# Actions Permissions Implementation Notes
Reading through #24635 and related PRs.
Need to understand why #23729 and #24554 were rejected.
Key points:
- Security first
- Org/repo boundaries
- No blanket permissions

View File

@ -0,0 +1,226 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package actions
import (
"context"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/modules/timeutil"
)
// PermissionMode represents the permission configuration mode
type PermissionMode int
const (
// PermissionModeRestricted - minimal permissions (default, secure)
PermissionModeRestricted PermissionMode = 0
// PermissionModePermissive - broad permissions (convenience)
PermissionModePermissive PermissionMode = 1
// PermissionModeCustom - user-defined permissions
PermissionModeCustom PermissionMode = 2
)
// ActionTokenPermission represents repository-level Actions token permissions
type ActionTokenPermission struct {
ID int64 `xorm:"pk autoincr"`
RepoID int64 `xorm:"UNIQUE NOT NULL"`
PermissionMode PermissionMode `xorm:"NOT NULL DEFAULT 0"`
// Granular permissions (only used in Custom mode)
ActionsRead bool `xorm:"NOT NULL DEFAULT false"`
ActionsWrite bool `xorm:"NOT NULL DEFAULT false"`
ContentsRead bool `xorm:"NOT NULL DEFAULT true"`
ContentsWrite bool `xorm:"NOT NULL DEFAULT false"`
IssuesRead bool `xorm:"NOT NULL DEFAULT false"`
IssuesWrite bool `xorm:"NOT NULL DEFAULT false"`
PackagesRead bool `xorm:"NOT NULL DEFAULT false"`
PackagesWrite bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsRead bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsWrite bool `xorm:"NOT NULL DEFAULT false"`
MetadataRead bool `xorm:"NOT NULL DEFAULT true"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
}
// ActionOrgPermission represents organization-level Actions token permissions
type ActionOrgPermission struct {
ID int64 `xorm:"pk autoincr"`
OrgID int64 `xorm:"UNIQUE NOT NULL"`
PermissionMode PermissionMode `xorm:"NOT NULL DEFAULT 0"`
AllowRepoOverride bool `xorm:"NOT NULL DEFAULT true"`
// Granular permissions (only used in Custom mode)
ActionsRead bool `xorm:"NOT NULL DEFAULT false"`
ActionsWrite bool `xorm:"NOT NULL DEFAULT false"`
ContentsRead bool `xorm:"NOT NULL DEFAULT true"`
ContentsWrite bool `xorm:"NOT NULL DEFAULT false"`
IssuesRead bool `xorm:"NOT NULL DEFAULT false"`
IssuesWrite bool `xorm:"NOT NULL DEFAULT false"`
PackagesRead bool `xorm:"NOT NULL DEFAULT false"`
PackagesWrite bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsRead bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsWrite bool `xorm:"NOT NULL DEFAULT false"`
MetadataRead bool `xorm:"NOT NULL DEFAULT true"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
}
func init() {
db.RegisterModel(new(ActionTokenPermission))
db.RegisterModel(new(ActionOrgPermission))
}
// GetRepoActionPermissions retrieves the Actions permissions for a repository
// If no configuration exists, returns nil (will use defaults)
func GetRepoActionPermissions(ctx context.Context, repoID int64) (*ActionTokenPermission, error) {
perm := &ActionTokenPermission{RepoID: repoID}
has, err := db.GetEngine(ctx).Get(perm)
if err != nil {
return nil, err
}
if !has {
return nil, nil // No custom config, will use defaults
}
return perm, nil
}
// GetOrgActionPermissions retrieves the Actions permissions for an organization
func GetOrgActionPermissions(ctx context.Context, orgID int64) (*ActionOrgPermission, error) {
perm := &ActionOrgPermission{OrgID: orgID}
has, err := db.GetEngine(ctx).Get(perm)
if err != nil {
return nil, err
}
if !has {
return nil, nil // No custom config, will use defaults
}
return perm, nil
}
// CreateOrUpdateRepoPermissions creates or updates repository-level permissions
func CreateOrUpdateRepoPermissions(ctx context.Context, perm *ActionTokenPermission) error {
existing := &ActionTokenPermission{RepoID: perm.RepoID}
has, err := db.GetEngine(ctx).Get(existing)
if err != nil {
return err
}
if has {
// Update existing
perm.ID = existing.ID
perm.CreatedUnix = existing.CreatedUnix
_, err = db.GetEngine(ctx).ID(perm.ID).Update(perm)
return err
}
// Create new
_, err = db.GetEngine(ctx).Insert(perm)
return err
}
// CreateOrUpdateOrgPermissions creates or updates organization-level permissions
func CreateOrUpdateOrgPermissions(ctx context.Context, perm *ActionOrgPermission) error {
existing := &ActionOrgPermission{OrgID: perm.OrgID}
has, err := db.GetEngine(ctx).Get(existing)
if err != nil {
return err
}
if has {
// Update existing
perm.ID = existing.ID
perm.CreatedUnix = existing.CreatedUnix
_, err = db.GetEngine(ctx).ID(perm.ID).Update(perm)
return err
}
// Create new
_, err = db.GetEngine(ctx).Insert(perm)
return err
}
// ToPermissionMap converts permission struct to a map for easy access
func (p *ActionTokenPermission) ToPermissionMap() map[string]map[string]bool {
// Apply permission mode defaults
var perms map[string]map[string]bool
switch p.PermissionMode {
case PermissionModeRestricted:
// Minimal permissions - only read metadata and contents
perms = map[string]map[string]bool{
"actions": {"read": false, "write": false},
"contents": {"read": true, "write": false},
"issues": {"read": false, "write": false},
"packages": {"read": false, "write": false},
"pull_requests": {"read": false, "write": false},
"metadata": {"read": true, "write": false},
}
case PermissionModePermissive:
// Broad permissions - read/write for most things
perms = map[string]map[string]bool{
"actions": {"read": true, "write": true},
"contents": {"read": true, "write": true},
"issues": {"read": true, "write": true},
"packages": {"read": true, "write": true},
"pull_requests": {"read": true, "write": true},
"metadata": {"read": true, "write": false},
}
case PermissionModeCustom:
// Use explicitly set permissions
perms = map[string]map[string]bool{
"actions": {"read": p.ActionsRead, "write": p.ActionsWrite},
"contents": {"read": p.ContentsRead, "write": p.ContentsWrite},
"issues": {"read": p.IssuesRead, "write": p.IssuesWrite},
"packages": {"read": p.PackagesRead, "write": p.PackagesWrite},
"pull_requests": {"read": p.PullRequestsRead, "write": p.PullRequestsWrite},
"metadata": {"read": p.MetadataRead, "write": false},
}
}
return perms
}
// ToPermissionMap converts org permission struct to a map
func (p *ActionOrgPermission) ToPermissionMap() map[string]map[string]bool {
var perms map[string]map[string]bool
switch p.PermissionMode {
case PermissionModeRestricted:
perms = map[string]map[string]bool{
"actions": {"read": false, "write": false},
"contents": {"read": true, "write": false},
"issues": {"read": false, "write": false},
"packages": {"read": false, "write": false},
"pull_requests": {"read": false, "write": false},
"metadata": {"read": true, "write": false},
}
case PermissionModePermissive:
perms = map[string]map[string]bool{
"actions": {"read": true, "write": true},
"contents": {"read": true, "write": true},
"issues": {"read": true, "write": true},
"packages": {"read": true, "write": true},
"pull_requests": {"read": true, "write": true},
"metadata": {"read": true, "write": false},
}
case PermissionModeCustom:
perms = map[string]map[string]bool{
"actions": {"read": p.ActionsRead, "write": p.ActionsWrite},
"contents": {"read": p.ContentsRead, "write": p.ContentsWrite},
"issues": {"read": p.IssuesRead, "write": p.IssuesWrite},
"packages": {"read": p.PackagesRead, "write": p.PackagesWrite},
"pull_requests": {"read": p.PullRequestsRead, "write": p.PullRequestsWrite},
"metadata": {"read": p.MetadataRead, "write": false},
}
}
return perms
}

View File

@ -0,0 +1,249 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package actions
import (
"context"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/modules/timeutil"
)
// ActionCrossRepoAccess represents cross-repository access rules
type ActionCrossRepoAccess struct {
ID int64 `xorm:"pk autoincr"`
OrgID int64 `xorm:"INDEX NOT NULL"`
SourceRepoID int64 `xorm:"INDEX NOT NULL"` // Repo that wants access
TargetRepoID int64 `xorm:"INDEX NOT NULL"` // Repo being accessed
// Access level: 0=none, 1=read, 2=write
AccessLevel int `xorm:"NOT NULL DEFAULT 0"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
}
// PackageRepoLink links packages to repositories
type PackageRepoLink struct {
ID int64 `xorm:"pk autoincr"`
PackageID int64 `xorm:"INDEX NOT NULL"`
RepoID int64 `xorm:"INDEX NOT NULL"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
}
func init() {
db.RegisterModel(new(ActionCrossRepoAccess))
db.RegisterModel(new(PackageRepoLink))
}
// ListCrossRepoAccessRules lists all cross-repo access rules for an organization
func ListCrossRepoAccessRules(ctx context.Context, orgID int64) ([]*ActionCrossRepoAccess, error) {
rules := make([]*ActionCrossRepoAccess, 0, 10)
err := db.GetEngine(ctx).
Where("org_id = ?", orgID).
Find(&rules)
return rules, err
}
// GetCrossRepoAccessByID retrieves a specific cross-repo access rule
func GetCrossRepoAccessByID(ctx context.Context, id int64) (*ActionCrossRepoAccess, error) {
rule := &ActionCrossRepoAccess{ID: id}
has, err := db.GetEngine(ctx).Get(rule)
if err != nil {
return nil, err
}
if !has {
return nil, db.ErrNotExist{Resource: "cross_repo_access", ID: id}
}
return rule, nil
}
// CheckCrossRepoAccess checks if source repo can access target repo
// Returns access level: 0=none, 1=read, 2=write
func CheckCrossRepoAccess(ctx context.Context, sourceRepoID, targetRepoID int64) (int, error) {
// If accessing same repo, always allow
// This is an optimization - no need to check rules
if sourceRepoID == targetRepoID {
return 2, nil // Full access to own repo
}
rule := &ActionCrossRepoAccess{}
has, err := db.GetEngine(ctx).
Where("source_repo_id = ? AND target_repo_id = ?", sourceRepoID, targetRepoID).
Get(rule)
if err != nil {
return 0, err
}
if !has {
// No rule found - deny access by default (secure default)
// This is intentional - cross-repo access must be explicitly granted
return 0, nil
}
return rule.AccessLevel, nil
}
// CreateCrossRepoAccess creates a new cross-repo access rule
func CreateCrossRepoAccess(ctx context.Context, rule *ActionCrossRepoAccess) error {
// Check if rule already exists
// We don't want duplicate rules for the same source-target pair
existing := &ActionCrossRepoAccess{}
has, err := db.GetEngine(ctx).
Where("org_id = ? AND source_repo_id = ? AND target_repo_id = ?",
rule.OrgID, rule.SourceRepoID, rule.TargetRepoID).
Get(existing)
if err != nil {
return err
}
if has {
// Update existing rule instead of creating duplicate
existing.AccessLevel = rule.AccessLevel
_, err = db.GetEngine(ctx).ID(existing.ID).Update(existing)
return err
}
// Create new rule
_, err = db.GetEngine(ctx).Insert(rule)
return err
}
// DeleteCrossRepoAccess deletes a cross-repo access rule
func DeleteCrossRepoAccess(ctx context.Context, id int64) error {
_, err := db.GetEngine(ctx).ID(id).Delete(&ActionCrossRepoAccess{})
return err
}
// Package-Repository Link Functions
// LinkPackageToRepo creates a link between a package and repository
// This allows Actions from that repository to access the package
func LinkPackageToRepo(ctx context.Context, packageID, repoID int64) error {
// Check if link already exists
existing := &PackageRepoLink{}
has, err := db.GetEngine(ctx).
Where("package_id = ? AND repo_id = ?", packageID, repoID).
Get(existing)
if err != nil {
return err
}
if has {
// Already linked - this is idempotent
return nil
}
link := &PackageRepoLink{
PackageID: packageID,
RepoID: repoID,
}
_, err = db.GetEngine(ctx).Insert(link)
return err
}
// UnlinkPackageFromRepo removes a link between package and repository
func UnlinkPackageFromRepo(ctx context.Context, packageID, repoID int64) error {
_, err := db.GetEngine(ctx).
Where("package_id = ? AND repo_id = ?", packageID, repoID).
Delete(&PackageRepoLink{})
return err
}
// IsPackageLinkedToRepo checks if a package is linked to a repository
func IsPackageLinkedToRepo(ctx context.Context, packageID, repoID int64) (bool, error) {
return db.GetEngine(ctx).
Where("package_id = ? AND repo_id = ?", packageID, repoID).
Exist(&PackageRepoLink{})
}
// GetPackageLinkedRepos returns all repos linked to a package
func GetPackageLinkedRepos(ctx context.Context, packageID int64) ([]int64, error) {
links := make([]*PackageRepoLink, 0, 10)
err := db.GetEngine(ctx).
Where("package_id = ?", packageID).
Find(&links)
if err != nil {
return nil, err
}
repoIDs := make([]int64, len(links))
for i, link := range links {
repoIDs[i] = link.RepoID
}
return repoIDs, nil
}
// GetRepoLinkedPackages returns all packages linked to a repository
func GetRepoLinkedPackages(ctx context.Context, repoID int64) ([]int64, error) {
links := make([]*PackageRepoLink, 0, 10)
err := db.GetEngine(ctx).
Where("repo_id = ?", repoID).
Find(&links)
if err != nil {
return nil, err
}
packageIDs := make([]int64, len(links))
for i, link := range links {
packageIDs[i] = link.PackageID
}
return packageIDs, nil
}
// CanAccessPackage checks if a repository's Actions can access a package
//
// Access is granted if ANY of these conditions are met:
// 1. Package is directly linked to the repository
// 2. Package is linked to another repo that allows cross-repo access to this repo
//
// This implements the security model from:
// https://github.com/go-gitea/gitea/issues/24635
func CanAccessPackage(ctx context.Context, repoID, packageID int64, needWrite bool) (bool, error) {
// Check direct linking
linked, err := IsPackageLinkedToRepo(ctx, packageID, repoID)
if err != nil {
return false, err
}
if linked {
// Package is directly linked - access granted!
// Note: Direct linking grants both read and write access
// This is intentional - if you link a package to your repo,
// you probably want to be able to publish to it
return true, nil
}
// Check indirect access via cross-repo rules
// Get all repos linked to this package
linkedRepos, err := GetPackageLinkedRepos(ctx, packageID)
if err != nil {
return false, err
}
// Check if we have cross-repo access to any of those repos
for _, targetRepoID := range linkedRepos {
accessLevel, err := CheckCrossRepoAccess(ctx, repoID, targetRepoID)
if err != nil {
continue // Skip on error, check next repo
}
if accessLevel > 0 {
// We have some level of access to the target repo
if needWrite && accessLevel < 2 {
// We need write but only have read - not enough
continue
}
// Access granted via cross-repo rule!
return true, nil
}
}
// No access found
return false, nil
}

View File

@ -26,6 +26,7 @@ import (
"code.gitea.io/gitea/models/migrations/v1_24"
"code.gitea.io/gitea/models/migrations/v1_25"
"code.gitea.io/gitea/models/migrations/v1_26"
"code.gitea.io/gitea/models/migrations/v1_27"
"code.gitea.io/gitea/models/migrations/v1_6"
"code.gitea.io/gitea/models/migrations/v1_7"
"code.gitea.io/gitea/models/migrations/v1_8"
@ -398,6 +399,7 @@ func prepareMigrationTasks() []*migration {
// Gitea 1.25.0 ends at migration ID number 322 (database version 323)
newMigration(323, "Add support for actions concurrency", v1_26.AddActionsConcurrency),
newMigration(324, "Add Actions token permissions configuration", v1_27.AddActionsPermissionsTables),
}
return preparedMigrations
}

View File

@ -0,0 +1,109 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package v1_27 //nolint
import (
"code.gitea.io/gitea/modules/timeutil"
"xorm.io/xorm"
)
// ActionTokenPermission represents the permissions configuration for Actions tokens at repository level
type ActionTokenPermission struct {
ID int64 `xorm:"pk autoincr"`
RepoID int64 `xorm:"UNIQUE NOT NULL"`
// Permission mode: 0=restricted (default), 1=permissive, 2=custom
PermissionMode int `xorm:"NOT NULL DEFAULT 0"`
// Individual permission flags (only used when PermissionMode=2/custom)
ActionsRead bool `xorm:"NOT NULL DEFAULT false"`
ActionsWrite bool `xorm:"NOT NULL DEFAULT false"`
ContentsRead bool `xorm:"NOT NULL DEFAULT true"` // Always true for basic functionality
ContentsWrite bool `xorm:"NOT NULL DEFAULT false"`
IssuesRead bool `xorm:"NOT NULL DEFAULT false"`
IssuesWrite bool `xorm:"NOT NULL DEFAULT false"`
PackagesRead bool `xorm:"NOT NULL DEFAULT false"`
PackagesWrite bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsRead bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsWrite bool `xorm:"NOT NULL DEFAULT false"`
MetadataRead bool `xorm:"NOT NULL DEFAULT true"` // Always true
CreatedUnix timeutil.TimeStamp `xorm:"created"`
UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
}
// ActionOrgPermission represents the permissions configuration for Actions tokens at organization level
type ActionOrgPermission struct {
ID int64 `xorm:"pk autoincr"`
OrgID int64 `xorm:"UNIQUE NOT NULL"`
// Permission mode: 0=restricted (default), 1=permissive, 2=custom
PermissionMode int `xorm:"NOT NULL DEFAULT 0"`
// Whether repos can override (set their own permissions)
// If false, all repos must use org settings
AllowRepoOverride bool `xorm:"NOT NULL DEFAULT true"`
// Individual permission flags (only used when PermissionMode=2/custom)
ActionsRead bool `xorm:"NOT NULL DEFAULT false"`
ActionsWrite bool `xorm:"NOT NULL DEFAULT false"`
ContentsRead bool `xorm:"NOT NULL DEFAULT true"`
ContentsWrite bool `xorm:"NOT NULL DEFAULT false"`
IssuesRead bool `xorm:"NOT NULL DEFAULT false"`
IssuesWrite bool `xorm:"NOT NULL DEFAULT false"`
PackagesRead bool `xorm:"NOT NULL DEFAULT false"`
PackagesWrite bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsRead bool `xorm:"NOT NULL DEFAULT false"`
PullRequestsWrite bool `xorm:"NOT NULL DEFAULT false"`
MetadataRead bool `xorm:"NOT NULL DEFAULT true"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
}
// ActionCrossRepoAccess represents cross-repository access rules within an organization
type ActionCrossRepoAccess struct {
ID int64 `xorm:"pk autoincr"`
OrgID int64 `xorm:"INDEX NOT NULL"`
SourceRepoID int64 `xorm:"INDEX NOT NULL"` // Repo that wants to access
TargetRepoID int64 `xorm:"INDEX NOT NULL"` // Repo being accessed
// Access level: 0=none, 1=read, 2=write
AccessLevel int `xorm:"NOT NULL DEFAULT 0"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
}
// PackageRepoLink links packages to repositories for permission checking
type PackageRepoLink struct {
ID int64 `xorm:"pk autoincr"`
PackageID int64 `xorm:"INDEX NOT NULL"`
RepoID int64 `xorm:"INDEX NOT NULL"`
CreatedUnix timeutil.TimeStamp `xorm:"created"`
}
func AddActionsPermissionsTables(x *xorm.Engine) error {
// Create action_token_permission table
if err := x.Sync2(new(ActionTokenPermission)); err != nil {
return err
}
// Create action_org_permission table
if err := x.Sync2(new(ActionOrgPermission)); err != nil {
return err
}
// Create action_cross_repo_access table
if err := x.Sync2(new(ActionCrossRepoAccess)); err != nil {
return err
}
// Create package_repo_link table
if err := x.Sync2(new(PackageRepoLink)); err != nil {
return err
}
return nil
}

View File

@ -0,0 +1,274 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package actions
import (
"context"
"fmt"
actions_model "code.gitea.io/gitea/models/actions"
)
// EffectivePermissions represents the final calculated permissions for an Actions token
type EffectivePermissions struct {
// Map structure: resource -> action -> allowed
// Example: {"contents": {"read": true, "write": false}}
Permissions map[string]map[string]bool
// Whether this token is from a fork PR (always restricted)
IsFromForkPR bool
// The permission mode used
Mode actions_model.PermissionMode
}
// PermissionChecker handles all permission checking logic for Actions tokens
type PermissionChecker struct {
ctx context.Context
}
// NewPermissionChecker creates a new permission checker
func NewPermissionChecker(ctx context.Context) *PermissionChecker {
return &PermissionChecker{ctx: ctx}
}
// GetEffectivePermissions calculates the final permissions for an Actions workflow
//
// Permission hierarchy (most restrictive wins):
// 1. Fork PR restriction (if applicable) - ALWAYS read-only
// 2. Organization settings (if exists) - caps maximum permissions
// 3. Repository settings (if exists) - further restricts
// 4. Workflow file permissions (if declared) - selects subset
//
// This implements the security model proposed in:
// https://github.com/go-gitea/gitea/issues/24635
func (pc *PermissionChecker) GetEffectivePermissions(
repoID int64,
orgID int64,
isFromForkPR bool,
workflowPermissions map[string]string, // From workflow YAML
) (*EffectivePermissions, error) {
// SECURITY: Fork PRs are ALWAYS restricted, regardless of any configuration
// This prevents malicious PRs from accessing sensitive resources
// Reference: https://github.com/go-gitea/gitea/pull/24554#issuecomment-1537040811
if isFromForkPR {
return &EffectivePermissions{
Permissions: getRestrictedPermissions(),
IsFromForkPR: true,
Mode: actions_model.PermissionModeRestricted,
}, nil
}
// Start with repository permissions (or defaults)
repoPerms, err := pc.getRepoPermissions(repoID)
if err != nil {
return nil, fmt.Errorf("failed to get repo permissions: %w", err)
}
// Apply organization cap if org exists
if orgID > 0 {
orgPerms, err := pc.getOrgPermissions(orgID)
if err != nil {
return nil, fmt.Errorf("failed to get org permissions: %w", err)
}
// Organization settings cap repository settings
// Repo can only reduce permissions, never increase beyond org
repoPerms = capPermissions(repoPerms, orgPerms)
}
// Apply workflow file permissions if specified
// Workflow can select a subset but cannot escalate beyond repo/org
finalPerms := repoPerms
if len(workflowPermissions) > 0 {
finalPerms = applyWorkflowPermissions(repoPerms, workflowPermissions)
}
return &EffectivePermissions{
Permissions: finalPerms,
IsFromForkPR: false,
Mode: actions_model.PermissionModeCustom, // Effective mode after merging
}, nil
}
// getRepoPermissions retrieves repository-level permissions or returns defaults
func (pc *PermissionChecker) getRepoPermissions(repoID int64) (map[string]map[string]bool, error) {
perm, err := actions_model.GetRepoActionPermissions(pc.ctx, repoID)
if err != nil {
return nil, err
}
if perm == nil {
// No custom config - use restricted defaults
return getRestrictedPermissions(), nil
}
return perm.ToPermissionMap(), nil
}
// getOrgPermissions retrieves organization-level permissions or returns defaults
func (pc *PermissionChecker) getOrgPermissions(orgID int64) (map[string]map[string]bool, error) {
perm, err := actions_model.GetOrgActionPermissions(pc.ctx, orgID)
if err != nil {
return nil, err
}
if perm == nil {
// No custom config - use restricted defaults
return getRestrictedPermissions(), nil
}
return perm.ToPermissionMap(), nil
}
// getRestrictedPermissions returns the default restricted permission set
func getRestrictedPermissions() map[string]map[string]bool {
return map[string]map[string]bool{
"actions": {"read": false, "write": false},
"contents": {"read": true, "write": false}, // Can read code
"issues": {"read": false, "write": false},
"packages": {"read": false, "write": false},
"pull_requests": {"read": false, "write": false},
"metadata": {"read": true, "write": false}, // Can read repo metadata
}
}
// capPermissions applies organizational caps to repository permissions
// Returns the more restrictive of the two permission sets
func capPermissions(repoPerms, orgPerms map[string]map[string]bool) map[string]map[string]bool {
result := make(map[string]map[string]bool)
for resource, actions := range repoPerms {
result[resource] = make(map[string]bool)
for action, repoAllowed := range actions {
orgAllowed := false
if orgActions, ok := orgPerms[resource]; ok {
orgAllowed = orgActions[action]
}
// Use the MORE restrictive (logical AND)
// If either org or repo denies, final result is deny
result[resource][action] = repoAllowed && orgAllowed
}
}
return result
}
// applyWorkflowPermissions applies workflow file permission declarations
// Workflow can only select a subset, cannot escalate
func applyWorkflowPermissions(basePerms map[string]map[string]bool, workflowPerms map[string]string) map[string]map[string]bool {
result := make(map[string]map[string]bool)
for resource := range basePerms {
result[resource] = make(map[string]bool)
// Check if workflow declares this resource
workflowPerm, declared := workflowPerms[resource]
if !declared {
// Not declared in workflow - use base permissions
result[resource] = basePerms[resource]
continue
}
// Workflow declared this resource - apply restrictions
switch workflowPerm {
case "none":
// Workflow explicitly denies
result[resource]["read"] = false
result[resource]["write"] = false
case "read":
// Workflow wants read - but only if base allows
result[resource]["read"] = basePerms[resource]["read"]
result[resource]["write"] = false
case "write":
// Workflow wants write - but only if base allows both read and write
// (write implies read in GitHub's model)
result[resource]["read"] = basePerms[resource]["read"]
result[resource]["write"] = basePerms[resource]["write"]
default:
// Unknown permission level - deny
result[resource]["read"] = false
result[resource]["write"] = false
}
}
return result
}
// CheckPermission checks if a specific action is allowed
func (ep *EffectivePermissions) CheckPermission(resource, action string) bool {
if ep.Permissions == nil {
return false
}
if actions, ok := ep.Permissions[resource]; ok {
return actions[action]
}
return false
}
// CanRead checks if reading a resource is allowed
func (ep *EffectivePermissions) CanRead(resource string) bool {
return ep.CheckPermission(resource, "read")
}
// CanWrite checks if writing to a resource is allowed
func (ep *EffectivePermissions) CanWrite(resource string) bool {
return ep.CheckPermission(resource, "write")
}
// ToTokenClaims converts permissions to JWT claims format
func (ep *EffectivePermissions) ToTokenClaims() map[string]interface{} {
claims := make(map[string]interface{})
// Add permissions map
claims["permissions"] = ep.Permissions
// Add fork PR flag
claims["is_fork_pr"] = ep.IsFromForkPR
// Add permission mode
claims["permission_mode"] = int(ep.Mode)
return claims
}
// ParsePermissionsFromClaims extracts permissions from JWT token claims
func ParsePermissionsFromClaims(claims map[string]interface{}) *EffectivePermissions {
ep := &EffectivePermissions{
Permissions: make(map[string]map[string]bool),
}
// Extract permissions map
if perms, ok := claims["permissions"].(map[string]interface{}); ok {
for resource, actions := range perms {
ep.Permissions[resource] = make(map[string]bool)
if actionMap, ok := actions.(map[string]interface{}); ok {
for action, allowed := range actionMap {
if allowedBool, ok := allowed.(bool); ok {
ep.Permissions[resource][action] = allowedBool
}
}
}
}
}
// Extract fork PR flag
if isForkPR, ok := claims["is_fork_pr"].(bool); ok {
ep.IsFromForkPR = isForkPR
}
// Extract permission mode
if mode, ok := claims["permission_mode"].(float64); ok {
ep.Mode = actions_model.PermissionMode(int(mode))
}
return ep
}

View File

@ -0,0 +1,231 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package actions
import (
"testing"
actions_model "code.gitea.io/gitea/models/actions"
"github.com/stretchr/testify/assert"
)
// TestGetEffectivePermissions_ForkPRAlwaysRestricted verifies that fork PRs
// are always restricted regardless of repo or org settings.
// This is critical for security - we don't want malicious forks to gain elevated
// permissions just by opening a PR. See the discussion in:
// https://github.com/go-gitea/gitea/pull/24554#issuecomment-1537040811
func TestGetEffectivePermissions_ForkPRAlwaysRestricted(t *testing.T) {
// Even if repo has permissive mode enabled
repoPerms := map[string]map[string]bool{
"contents": {"read": true, "write": true},
"packages": {"read": true, "write": true},
"issues": {"read": true, "write": true},
}
// Fork PR should still be read-only
result := applyForkPRRestrictions(repoPerms)
assert.True(t, result["contents"]["read"], "Should allow reading contents")
assert.False(t, result["contents"]["write"], "Should NOT allow writing contents")
assert.False(t, result["packages"]["write"], "Should NOT allow package writes")
assert.False(t, result["issues"]["write"], "Should NOT allow issue writes")
}
// TestOrgPermissionsCap verifies that organization settings act as a ceiling
// for repository settings. Repos can be more restrictive but not more permissive.
func TestOrgPermissionsCap(t *testing.T) {
// Org says: no package writes
orgPerms := map[string]map[string]bool{
"packages": {"read": true, "write": false},
"contents": {"read": true, "write": true},
}
// Repo tries to enable package writes
repoPerms := map[string]map[string]bool{
"packages": {"read": true, "write": true}, // Trying to override!
"contents": {"read": true, "write": true},
}
result := capPermissions(repoPerms, orgPerms)
// Org restriction should win
assert.False(t, result["packages"]["write"], "Org should prevent package writes")
assert.True(t, result["contents"]["write"], "Contents write should be allowed")
}
// TestWorkflowCannotEscalate verifies that workflow file declarations
// cannot grant more permissions than repo/org settings allow.
// This is important because in Gitea, anyone with write access can edit workflows
// (unlike GitHub which has CODEOWNERS protection).
func TestWorkflowCannotEscalate(t *testing.T) {
// Base permissions: read-only for packages
basePerms := map[string]map[string]bool{
"packages": {"read": true, "write": false},
"contents": {"read": true, "write": true},
}
// Workflow tries to declare package write
workflowPerms := map[string]string{
"packages": "write", // Trying to escalate!
"contents": "write",
}
result := applyWorkflowPermissions(basePerms, workflowPerms)
// Should NOT be able to escalate
assert.False(t, result["packages"]["write"], "Workflow should not escalate package perms")
assert.True(t, result["contents"]["write"], "Contents write should still work")
}
// TestWorkflowCanReducePermissions verifies that workflows CAN reduce permissions
// This is useful for defense-in-depth - even if repo has broad permissions,
// a specific workflow can declare it only needs minimal permissions.
func TestWorkflowCanReducePermissions(t *testing.T) {
// Base permissions: write access
basePerms := map[string]map[string]bool{
"contents": {"read": true, "write": true},
"issues": {"read": true, "write": true},
}
// Workflow declares it only needs read
workflowPerms := map[string]string{
"contents": "read",
"issues": "none", // Explicitly denies
}
result := applyWorkflowPermissions(basePerms, workflowPerms)
assert.True(t, result["contents"]["read"], "Should allow reading")
assert.False(t, result["contents"]["write"], "Should reduce to read-only")
assert.False(t, result["issues"]["read"], "Should deny issues entirely")
}
// TestRestrictedModeDefaults verifies that restricted mode has sensible defaults
// We want it to be usable (can clone code, read metadata) but secure (no writes)
func TestRestrictedModeDefaults(t *testing.T) {
perms := getRestrictedPermissions()
// Should be able to read code (needed for checkout action)
assert.True(t, perms["contents"]["read"], "Must be able to read code")
assert.True(t, perms["metadata"]["read"], "Must be able to read metadata")
// Should NOT be able to write anything
assert.False(t, perms["contents"]["write"], "Should not write code")
assert.False(t, perms["packages"]["write"], "Should not write packages")
assert.False(t, perms["issues"]["write"], "Should not write issues")
}
// TestPermissionModeTransitions tests that changing modes works correctly
// This is important for the UI - users should be able to switch modes easily
func TestPermissionModeTransitions(t *testing.T) {
tests := []struct {
name string
mode actions_model.PermissionMode
expectPackageWrite bool
expectContentsWrite bool
}{
{
name: "Restricted mode - no writes",
mode: actions_model.PermissionModeRestricted,
expectPackageWrite: false,
expectContentsWrite: false,
},
{
name: "Permissive mode - has writes",
mode: actions_model.PermissionModePermissive,
expectPackageWrite: true,
expectContentsWrite: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
perm := &actions_model.ActionTokenPermission{
PermissionMode: tt.mode,
}
permMap := perm.ToPermissionMap()
assert.Equal(t, tt.expectPackageWrite, permMap["packages"]["write"])
assert.Equal(t, tt.expectContentsWrite, permMap["contents"]["write"])
})
}
}
// TestMultipleLayers tests the full permission calculation with all layers
// This simulates a real-world scenario with org, repo, and workflow permissions
func TestMultipleLayers(t *testing.T) {
// Scenario: Org allows package reads, Repo allows package writes,
// but workflow only declares package read
orgPerms := map[string]map[string]bool{
"packages": {"read": true, "write": false}, // Org blocks writes
}
repoPerms := map[string]map[string]bool{
"packages": {"read": true, "write": true}, // Repo tries to enable
}
workflowPerms := map[string]string{
"packages": "read", // Workflow only needs read
}
// Apply caps (org limits repo)
afterOrgCap := capPermissions(repoPerms, orgPerms)
assert.False(t, afterOrgCap["packages"]["write"], "Org should block write")
// Apply workflow (workflow selects read)
final := applyWorkflowPermissions(afterOrgCap, workflowPerms)
assert.True(t, final["packages"]["read"], "Should have read access")
assert.False(t, final["packages"]["write"], "Should not have write (org blocked)")
}
// BenchmarkPermissionCalculation measures permission calculation performance
// This is important because permission checks happen on every API call with Actions tokens
// We want to ensure this doesn't become a bottleneck
func BenchmarkPermissionCalculation(b *testing.B) {
repoPerms := map[string]map[string]bool{
"actions": {"read": true, "write": false},
"contents": {"read": true, "write": true},
"issues": {"read": true, "write": true},
"packages": {"read": true, "write": true},
"pull_requests": {"read": true, "write": false},
"metadata": {"read": true, "write": false},
}
orgPerms := map[string]map[string]bool{
"actions": {"read": true, "write": false},
"contents": {"read": true, "write": false},
"issues": {"read": true, "write": false},
"packages": {"read": false, "write": false},
"pull_requests": {"read": true, "write": false},
"metadata": {"read": true, "write": false},
}
workflowPerms := map[string]string{
"contents": "read",
"packages": "read",
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
capped := capPermissions(repoPerms, orgPerms)
_ = applyWorkflowPermissions(capped, workflowPerms)
}
}
// Helper function for fork PR tests
// In real implementation, this would be in permission_checker.go
// TODO: Refactor this into the main codebase if these tests pass
func applyForkPRRestrictions(perms map[string]map[string]bool) map[string]map[string]bool {
// Fork PRs get read-only access to contents and metadata, nothing else
return map[string]map[string]bool{
"contents": {"read": true, "write": false},
"metadata": {"read": true, "write": false},
"actions": {"read": false, "write": false},
"packages": {"read": false, "write": false},
"issues": {"read": false, "write": false},
"pull_requests": {"read": false, "write": false},
}
}

View File

@ -5,12 +5,10 @@ package charset
import (
"bytes"
"fmt"
"io"
"strings"
"unicode/utf8"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
@ -23,60 +21,39 @@ import (
var UTF8BOM = []byte{'\xef', '\xbb', '\xbf'}
type ConvertOpts struct {
KeepBOM bool
KeepBOM bool
ErrorReplacement []byte
ErrorReturnOrigin bool
}
var ToUTF8WithFallbackReaderPrefetchSize = 16 * 1024
// ToUTF8WithFallbackReader detects the encoding of content and converts to UTF-8 reader if possible
func ToUTF8WithFallbackReader(rd io.Reader, opts ConvertOpts) io.Reader {
buf := make([]byte, 2048)
buf := make([]byte, ToUTF8WithFallbackReaderPrefetchSize)
n, err := util.ReadAtMost(rd, buf)
if err != nil {
return io.MultiReader(bytes.NewReader(MaybeRemoveBOM(buf[:n], opts)), rd)
}
charsetLabel, err := DetectEncoding(buf[:n])
if err != nil || charsetLabel == "UTF-8" {
return io.MultiReader(bytes.NewReader(MaybeRemoveBOM(buf[:n], opts)), rd)
}
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
// read error occurs, don't do any processing
return io.MultiReader(bytes.NewReader(buf[:n]), rd)
}
return transform.NewReader(
io.MultiReader(
bytes.NewReader(MaybeRemoveBOM(buf[:n], opts)),
rd,
),
encoding.NewDecoder(),
)
}
// ToUTF8 converts content to UTF8 encoding
func ToUTF8(content []byte, opts ConvertOpts) (string, error) {
charsetLabel, err := DetectEncoding(content)
if err != nil {
return "", err
} else if charsetLabel == "UTF-8" {
return string(MaybeRemoveBOM(content, opts)), nil
charsetLabel, _ := DetectEncoding(buf[:n])
if charsetLabel == "UTF-8" {
// is utf-8, try to remove BOM and read it as-is
return io.MultiReader(bytes.NewReader(maybeRemoveBOM(buf[:n], opts)), rd)
}
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
return string(content), fmt.Errorf("Unknown encoding: %s", charsetLabel)
// unknown charset, don't do any processing
return io.MultiReader(bytes.NewReader(buf[:n]), rd)
}
// If there is an error, we concatenate the nicely decoded part and the
// original left over. This way we won't lose much data.
result, n, err := transform.Bytes(encoding.NewDecoder(), content)
if err != nil {
result = append(result, content[n:]...)
}
result = MaybeRemoveBOM(result, opts)
return string(result), err
// convert from charset to utf-8
return transform.NewReader(
io.MultiReader(bytes.NewReader(buf[:n]), rd),
encoding.NewDecoder(),
)
}
// ToUTF8WithFallback detects the encoding of content and converts to UTF-8 if possible
@ -85,73 +62,84 @@ func ToUTF8WithFallback(content []byte, opts ConvertOpts) []byte {
return bs
}
// ToUTF8DropErrors makes sure the return string is valid utf-8; attempts conversion if possible
func ToUTF8DropErrors(content []byte, opts ConvertOpts) []byte {
charsetLabel, err := DetectEncoding(content)
if err != nil || charsetLabel == "UTF-8" {
return MaybeRemoveBOM(content, opts)
func ToUTF8DropErrors(content []byte) []byte {
return ToUTF8(content, ConvertOpts{ErrorReplacement: []byte{' '}})
}
func ToUTF8(content []byte, opts ConvertOpts) []byte {
charsetLabel, _ := DetectEncoding(content)
if charsetLabel == "UTF-8" {
return maybeRemoveBOM(content, opts)
}
encoding, _ := charset.Lookup(charsetLabel)
if encoding == nil {
setting.PanicInDevOrTesting("unsupported detected charset %q, it shouldn't happen", charsetLabel)
return content
}
// We ignore any non-decodable parts from the file.
// Some parts might be lost
var decoded []byte
decoder := encoding.NewDecoder()
idx := 0
for {
for idx < len(content) {
result, n, err := transform.Bytes(decoder, content[idx:])
decoded = append(decoded, result...)
if err == nil {
break
}
decoded = append(decoded, ' ')
idx = idx + n + 1
if idx >= len(content) {
break
if opts.ErrorReturnOrigin {
return content
}
if opts.ErrorReplacement == nil {
decoded = append(decoded, content[idx+n])
} else {
decoded = append(decoded, opts.ErrorReplacement...)
}
idx += n + 1
}
return MaybeRemoveBOM(decoded, opts)
return maybeRemoveBOM(decoded, opts)
}
// MaybeRemoveBOM removes a UTF-8 BOM from a []byte when opts.KeepBOM is false
func MaybeRemoveBOM(content []byte, opts ConvertOpts) []byte {
// maybeRemoveBOM removes a UTF-8 BOM from a []byte when opts.KeepBOM is false
func maybeRemoveBOM(content []byte, opts ConvertOpts) []byte {
if opts.KeepBOM {
return content
}
if len(content) > 2 && bytes.Equal(content[0:3], UTF8BOM) {
return content[3:]
}
return content
return bytes.TrimPrefix(content, UTF8BOM)
}
// DetectEncoding detect the encoding of content
func DetectEncoding(content []byte) (string, error) {
// it always returns a detected or guessed "encoding" string, no matter error happens or not
func DetectEncoding(content []byte) (encoding string, _ error) {
// First we check if the content represents valid utf8 content excepting a truncated character at the end.
// Now we could decode all the runes in turn but this is not necessarily the cheapest thing to do
// instead we walk backwards from the end to trim off a the incomplete character
// instead we walk backwards from the end to trim off the incomplete character
toValidate := content
end := len(toValidate) - 1
if end < 0 {
// no-op
} else if toValidate[end]>>5 == 0b110 {
// Incomplete 1 byte extension e.g. © <c2><a9> which has been truncated to <c2>
toValidate = toValidate[:end]
} else if end > 0 && toValidate[end]>>6 == 0b10 && toValidate[end-1]>>4 == 0b1110 {
// Incomplete 2 byte extension e.g. ⛔ <e2><9b><94> which has been truncated to <e2><9b>
toValidate = toValidate[:end-1]
} else if end > 1 && toValidate[end]>>6 == 0b10 && toValidate[end-1]>>6 == 0b10 && toValidate[end-2]>>3 == 0b11110 {
// Incomplete 3 byte extension e.g. 💩 <f0><9f><92><a9> which has been truncated to <f0><9f><92>
toValidate = toValidate[:end-2]
// U+0000 U+007F 0yyyzzzz
// U+0080 U+07FF 110xxxyy 10yyzzzz
// U+0800 U+FFFF 1110wwww 10xxxxyy 10yyzzzz
// U+010000 U+10FFFF 11110uvv 10vvwwww 10xxxxyy 10yyzzzz
cnt := 0
for end >= 0 && cnt < 4 {
c := toValidate[end]
if c>>5 == 0b110 || c>>4 == 0b1110 || c>>3 == 0b11110 {
// a leading byte
toValidate = toValidate[:end]
break
} else if c>>6 == 0b10 {
// a continuation byte
end--
} else {
// not an utf-8 byte
break
}
cnt++
}
if utf8.Valid(toValidate) {
log.Debug("Detected encoding: utf-8 (fast)")
return "UTF-8", nil
}
@ -160,7 +148,7 @@ func DetectEncoding(content []byte) (string, error) {
if len(content) < 1024 {
// Check if original content is valid
if _, err := textDetector.DetectBest(content); err != nil {
return "", err
return util.IfZero(setting.Repository.AnsiCharset, "UTF-8"), err
}
times := 1024 / len(content)
detectContent = make([]byte, 0, times*len(content))
@ -171,14 +159,10 @@ func DetectEncoding(content []byte) (string, error) {
detectContent = content
}
// Now we can't use DetectBest or just results[0] because the result isn't stable - so we need a tie break
// Now we can't use DetectBest or just results[0] because the result isn't stable - so we need a tie-break
results, err := textDetector.DetectAll(detectContent)
if err != nil {
if err == chardet.NotDetectedError && len(setting.Repository.AnsiCharset) > 0 {
log.Debug("Using default AnsiCharset: %s", setting.Repository.AnsiCharset)
return setting.Repository.AnsiCharset, nil
}
return "", err
return util.IfZero(setting.Repository.AnsiCharset, "UTF-8"), err
}
topConfidence := results[0].Confidence
@ -201,11 +185,9 @@ func DetectEncoding(content []byte) (string, error) {
}
// FIXME: to properly decouple this function the fallback ANSI charset should be passed as an argument
if topResult.Charset != "UTF-8" && len(setting.Repository.AnsiCharset) > 0 {
log.Debug("Using default AnsiCharset: %s", setting.Repository.AnsiCharset)
if topResult.Charset != "UTF-8" && setting.Repository.AnsiCharset != "" {
return setting.Repository.AnsiCharset, err
}
log.Debug("Detected encoding: %s", topResult.Charset)
return topResult.Charset, err
return topResult.Charset, nil
}

View File

@ -4,108 +4,89 @@
package charset
import (
"bytes"
"io"
"os"
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
"github.com/stretchr/testify/assert"
)
func resetDefaultCharsetsOrder() {
defaultDetectedCharsetsOrder := make([]string, 0, len(setting.Repository.DetectedCharsetsOrder))
for _, charset := range setting.Repository.DetectedCharsetsOrder {
defaultDetectedCharsetsOrder = append(defaultDetectedCharsetsOrder, strings.ToLower(strings.TrimSpace(charset)))
}
func TestMain(m *testing.M) {
setting.Repository.DetectedCharsetScore = map[string]int{}
i := 0
for _, charset := range defaultDetectedCharsetsOrder {
canonicalCharset := strings.ToLower(strings.TrimSpace(charset))
if _, has := setting.Repository.DetectedCharsetScore[canonicalCharset]; !has {
setting.Repository.DetectedCharsetScore[canonicalCharset] = i
i++
}
for i, charset := range setting.Repository.DetectedCharsetsOrder {
setting.Repository.DetectedCharsetScore[strings.ToLower(charset)] = i
}
os.Exit(m.Run())
}
func TestMaybeRemoveBOM(t *testing.T) {
res := MaybeRemoveBOM([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
res := maybeRemoveBOM([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
res = MaybeRemoveBOM([]byte{0xef, 0xbb, 0xbf, 0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
res = maybeRemoveBOM([]byte{0xef, 0xbb, 0xbf, 0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
}
func TestToUTF8(t *testing.T) {
resetDefaultCharsetsOrder()
// Note: golang compiler seems so behave differently depending on the current
// locale, so some conversions might behave differently. For that reason, we don't
// depend on particular conversions but in expected behaviors.
res, err := ToUTF8([]byte{0x41, 0x42, 0x43}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, "ABC", res)
res := ToUTF8([]byte{0x41, 0x42, 0x43}, ConvertOpts{})
assert.Equal(t, "ABC", string(res))
// "áéíóú"
res, err = ToUTF8([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, []byte(res))
res = ToUTF8([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
// "áéíóú"
res, err = ToUTF8([]byte{
res = ToUTF8([]byte{
0xef, 0xbb, 0xbf, 0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3,
0xc3, 0xba,
}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, []byte(res))
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
res, err = ToUTF8([]byte{
res = ToUTF8([]byte{
0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63,
0xF3, 0x6D, 0x6F, 0x20, 0xF1, 0x6F, 0x73, 0x41, 0x41, 0x41, 0x2e,
}, ConvertOpts{})
assert.NoError(t, err)
stringMustStartWith(t, "Hola,", res)
stringMustEndWith(t, "AAA.", res)
res, err = ToUTF8([]byte{
res = ToUTF8([]byte{
0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63,
0xF3, 0x6D, 0x6F, 0x20, 0x07, 0xA4, 0x6F, 0x73, 0x41, 0x41, 0x41, 0x2e,
}, ConvertOpts{})
assert.NoError(t, err)
stringMustStartWith(t, "Hola,", res)
stringMustEndWith(t, "AAA.", res)
res, err = ToUTF8([]byte{
res = ToUTF8([]byte{
0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63,
0xF3, 0x6D, 0x6F, 0x20, 0x81, 0xA4, 0x6F, 0x73, 0x41, 0x41, 0x41, 0x2e,
}, ConvertOpts{})
assert.NoError(t, err)
stringMustStartWith(t, "Hola,", res)
stringMustEndWith(t, "AAA.", res)
// Japanese (Shift-JIS)
// 日属秘ぞしちゅ。
res, err = ToUTF8([]byte{
res = ToUTF8([]byte{
0x93, 0xFA, 0x91, 0xAE, 0x94, 0xE9, 0x82, 0xBC, 0x82, 0xB5, 0x82,
0xBF, 0x82, 0xE3, 0x81, 0x42,
}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{
0xE6, 0x97, 0xA5, 0xE5, 0xB1, 0x9E, 0xE7, 0xA7, 0x98, 0xE3,
0x81, 0x9E, 0xE3, 0x81, 0x97, 0xE3, 0x81, 0xA1, 0xE3, 0x82, 0x85, 0xE3, 0x80, 0x82,
},
[]byte(res))
}, res)
res, err = ToUTF8([]byte{0x00, 0x00, 0x00, 0x00}, ConvertOpts{})
assert.NoError(t, err)
assert.Equal(t, []byte{0x00, 0x00, 0x00, 0x00}, []byte(res))
res = ToUTF8([]byte{0x00, 0x00, 0x00, 0x00}, ConvertOpts{})
assert.Equal(t, []byte{0x00, 0x00, 0x00, 0x00}, res)
}
func TestToUTF8WithFallback(t *testing.T) {
resetDefaultCharsetsOrder()
// "ABC"
res := ToUTF8WithFallback([]byte{0x41, 0x42, 0x43}, ConvertOpts{})
assert.Equal(t, []byte{0x41, 0x42, 0x43}, res)
@ -152,54 +133,58 @@ func TestToUTF8WithFallback(t *testing.T) {
}
func TestToUTF8DropErrors(t *testing.T) {
resetDefaultCharsetsOrder()
// "ABC"
res := ToUTF8DropErrors([]byte{0x41, 0x42, 0x43}, ConvertOpts{})
res := ToUTF8DropErrors([]byte{0x41, 0x42, 0x43})
assert.Equal(t, []byte{0x41, 0x42, 0x43}, res)
// "áéíóú"
res = ToUTF8DropErrors([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba})
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
// UTF8 BOM + "áéíóú"
res = ToUTF8DropErrors([]byte{0xef, 0xbb, 0xbf, 0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0xef, 0xbb, 0xbf, 0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba})
assert.Equal(t, []byte{0xc3, 0xa1, 0xc3, 0xa9, 0xc3, 0xad, 0xc3, 0xb3, 0xc3, 0xba}, res)
// "Hola, así cómo ños"
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0xF1, 0x6F, 0x73}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0xF1, 0x6F, 0x73})
assert.Equal(t, []byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73}, res[:8])
assert.Equal(t, []byte{0x73}, res[len(res)-1:])
// "Hola, así cómo "
minmatch := []byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xC3, 0xAD, 0x20, 0x63, 0xC3, 0xB3, 0x6D, 0x6F, 0x20}
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0x07, 0xA4, 0x6F, 0x73}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0x07, 0xA4, 0x6F, 0x73})
// Do not fail for differences in invalid cases, as the library might change the conversion criteria for those
assert.Equal(t, minmatch, res[0:len(minmatch)])
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0x81, 0xA4, 0x6F, 0x73}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0x48, 0x6F, 0x6C, 0x61, 0x2C, 0x20, 0x61, 0x73, 0xED, 0x20, 0x63, 0xF3, 0x6D, 0x6F, 0x20, 0x81, 0xA4, 0x6F, 0x73})
// Do not fail for differences in invalid cases, as the library might change the conversion criteria for those
assert.Equal(t, minmatch, res[0:len(minmatch)])
// Japanese (Shift-JIS)
// "日属秘ぞしちゅ。"
res = ToUTF8DropErrors([]byte{0x93, 0xFA, 0x91, 0xAE, 0x94, 0xE9, 0x82, 0xBC, 0x82, 0xB5, 0x82, 0xBF, 0x82, 0xE3, 0x81, 0x42}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0x93, 0xFA, 0x91, 0xAE, 0x94, 0xE9, 0x82, 0xBC, 0x82, 0xB5, 0x82, 0xBF, 0x82, 0xE3, 0x81, 0x42})
assert.Equal(t, []byte{
0xE6, 0x97, 0xA5, 0xE5, 0xB1, 0x9E, 0xE7, 0xA7, 0x98, 0xE3,
0x81, 0x9E, 0xE3, 0x81, 0x97, 0xE3, 0x81, 0xA1, 0xE3, 0x82, 0x85, 0xE3, 0x80, 0x82,
}, res)
res = ToUTF8DropErrors([]byte{0x00, 0x00, 0x00, 0x00}, ConvertOpts{})
res = ToUTF8DropErrors([]byte{0x00, 0x00, 0x00, 0x00})
assert.Equal(t, []byte{0x00, 0x00, 0x00, 0x00}, res)
}
func TestDetectEncoding(t *testing.T) {
resetDefaultCharsetsOrder()
testSuccess := func(b []byte, expected string) {
encoding, err := DetectEncoding(b)
assert.NoError(t, err)
assert.Equal(t, expected, encoding)
}
// invalid bytes
encoding, err := DetectEncoding([]byte{0xfa})
assert.Error(t, err)
assert.Equal(t, "UTF-8", encoding)
// utf-8
b := []byte("just some ascii")
testSuccess(b, "UTF-8")
@ -214,169 +199,49 @@ func TestDetectEncoding(t *testing.T) {
// iso-8859-1: d<accented e>cor<newline>
b = []byte{0x44, 0xe9, 0x63, 0x6f, 0x72, 0x0a}
encoding, err := DetectEncoding(b)
encoding, err = DetectEncoding(b)
assert.NoError(t, err)
assert.Contains(t, encoding, "ISO-8859-1")
old := setting.Repository.AnsiCharset
setting.Repository.AnsiCharset = "placeholder"
defer func() {
setting.Repository.AnsiCharset = old
}()
testSuccess(b, "placeholder")
// invalid bytes
b = []byte{0xfa}
_, err = DetectEncoding(b)
assert.Error(t, err)
defer test.MockVariableValue(&setting.Repository.AnsiCharset, "MyEncoding")()
testSuccess(b, "MyEncoding")
}
func stringMustStartWith(t *testing.T, expected, value string) {
assert.Equal(t, expected, value[:len(expected)])
func stringMustStartWith(t *testing.T, expected string, value []byte) {
assert.Equal(t, expected, string(value[:len(expected)]))
}
func stringMustEndWith(t *testing.T, expected, value string) {
assert.Equal(t, expected, value[len(value)-len(expected):])
func stringMustEndWith(t *testing.T, expected string, value []byte) {
assert.Equal(t, expected, string(value[len(value)-len(expected):]))
}
func TestToUTF8WithFallbackReader(t *testing.T) {
resetDefaultCharsetsOrder()
test.MockVariableValue(&ToUTF8WithFallbackReaderPrefetchSize)
for testLen := range 2048 {
pattern := " test { () }\n"
input := ""
for len(input) < testLen {
input += pattern
}
input = input[:testLen]
input += "// Выключаем"
rd := ToUTF8WithFallbackReader(bytes.NewReader([]byte(input)), ConvertOpts{})
block := "aá啊🤔"
runes := []rune(block)
assert.Len(t, string(runes[0]), 1)
assert.Len(t, string(runes[1]), 2)
assert.Len(t, string(runes[2]), 3)
assert.Len(t, string(runes[3]), 4)
content := strings.Repeat(block, 2)
for i := 1; i < len(content); i++ {
encoding, err := DetectEncoding([]byte(content[:i]))
assert.NoError(t, err)
assert.Equal(t, "UTF-8", encoding)
ToUTF8WithFallbackReaderPrefetchSize = i
rd := ToUTF8WithFallbackReader(strings.NewReader(content), ConvertOpts{})
r, _ := io.ReadAll(rd)
assert.Equalf(t, input, string(r), "testing string len=%d", testLen)
assert.Equal(t, content, string(r))
}
for _, r := range runes {
content = "abc abc " + string(r) + string(r) + string(r)
for i := 0; i < len(content); i++ {
encoding, err := DetectEncoding([]byte(content[:i]))
assert.NoError(t, err)
assert.Equal(t, "UTF-8", encoding)
}
}
truncatedOneByteExtension := failFastBytes
encoding, _ := DetectEncoding(truncatedOneByteExtension)
assert.Equal(t, "UTF-8", encoding)
truncatedTwoByteExtension := failFastBytes
truncatedTwoByteExtension[len(failFastBytes)-1] = 0x9b
truncatedTwoByteExtension[len(failFastBytes)-2] = 0xe2
encoding, _ = DetectEncoding(truncatedTwoByteExtension)
assert.Equal(t, "UTF-8", encoding)
truncatedThreeByteExtension := failFastBytes
truncatedThreeByteExtension[len(failFastBytes)-1] = 0x92
truncatedThreeByteExtension[len(failFastBytes)-2] = 0x9f
truncatedThreeByteExtension[len(failFastBytes)-3] = 0xf0
encoding, _ = DetectEncoding(truncatedThreeByteExtension)
assert.Equal(t, "UTF-8", encoding)
}
var failFastBytes = []byte{
0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x20, 0x6f, 0x72, 0x67, 0x2e, 0x61, 0x70, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x74, 0x6f,
0x6f, 0x6c, 0x73, 0x2e, 0x61, 0x6e, 0x74, 0x2e, 0x74, 0x61, 0x73, 0x6b, 0x64, 0x65, 0x66, 0x73, 0x2e, 0x63, 0x6f, 0x6e,
0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x4f, 0x73, 0x0a, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x20, 0x6f, 0x72, 0x67,
0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f,
0x74, 0x2e, 0x67, 0x72, 0x61, 0x64, 0x6c, 0x65, 0x2e, 0x74, 0x61, 0x73, 0x6b, 0x73, 0x2e, 0x72, 0x75, 0x6e, 0x2e, 0x42,
0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x0a, 0x0a, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x73, 0x20, 0x7b, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x64, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d,
0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x22, 0x29, 0x0a, 0x7d, 0x0a, 0x0a, 0x64, 0x65, 0x70, 0x65,
0x6e, 0x64, 0x65, 0x6e, 0x63, 0x69, 0x65, 0x73, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65,
0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x61, 0x70, 0x69, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d,
0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74,
0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x61, 0x70, 0x69, 0x2d, 0x64, 0x6f, 0x63, 0x73, 0x22, 0x29,
0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e,
0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x64, 0x62,
0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69,
0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65,
0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x3a, 0x69, 0x6e, 0x74, 0x65, 0x67, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2d, 0x66,
0x73, 0x22, 0x29, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x28, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x28, 0x22, 0x3a, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72,
0x3a, 0x69, 0x6e, 0x74, 0x65, 0x67, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2d, 0x6d, 0x71, 0x22, 0x29, 0x29, 0x0a, 0x0a,
0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22,
0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e,
0x2d, 0x61, 0x75, 0x74, 0x68, 0x2d, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2d, 0x73, 0x74, 0x61, 0x72, 0x74,
0x65, 0x72, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63,
0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d, 0x68, 0x61, 0x6c, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c,
0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a, 0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e,
0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d, 0x63, 0x6f, 0x72, 0x65, 0x22, 0x29, 0x0a,
0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28,
0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b,
0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d, 0x73, 0x74,
0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x77, 0x65, 0x62, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c,
0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69,
0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72,
0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d, 0x73, 0x74, 0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x61, 0x6f, 0x70,
0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f,
0x72, 0x6b, 0x2e, 0x62, 0x6f, 0x6f, 0x74, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x2d,
0x73, 0x74, 0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x61, 0x63, 0x74, 0x75, 0x61, 0x74, 0x6f, 0x72, 0x22, 0x29, 0x0a, 0x20,
0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f,
0x72, 0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63,
0x6c, 0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74,
0x61, 0x72, 0x74, 0x65, 0x72, 0x2d, 0x62, 0x6f, 0x6f, 0x74, 0x73, 0x74, 0x72, 0x61, 0x70, 0x22, 0x29, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72,
0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63, 0x6c,
0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74, 0x61,
0x72, 0x74, 0x65, 0x72, 0x2d, 0x63, 0x6f, 0x6e, 0x73, 0x75, 0x6c, 0x2d, 0x61, 0x6c, 0x6c, 0x22, 0x29, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72,
0x67, 0x2e, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x63, 0x6c,
0x6f, 0x75, 0x64, 0x3a, 0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x73, 0x74, 0x61,
0x72, 0x74, 0x65, 0x72, 0x2d, 0x73, 0x6c, 0x65, 0x75, 0x74, 0x68, 0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d,
0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6f, 0x72, 0x67, 0x2e, 0x73, 0x70,
0x72, 0x69, 0x6e, 0x67, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x77, 0x6f, 0x72, 0x6b, 0x2e, 0x72, 0x65, 0x74, 0x72, 0x79, 0x3a,
0x73, 0x70, 0x72, 0x69, 0x6e, 0x67, 0x2d, 0x72, 0x65, 0x74, 0x72, 0x79, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x63, 0x68, 0x2e, 0x71,
0x6f, 0x73, 0x2e, 0x6c, 0x6f, 0x67, 0x62, 0x61, 0x63, 0x6b, 0x3a, 0x6c, 0x6f, 0x67, 0x62, 0x61, 0x63, 0x6b, 0x2d, 0x63,
0x6c, 0x61, 0x73, 0x73, 0x69, 0x63, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d,
0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x69, 0x6f, 0x2e, 0x6d, 0x69, 0x63, 0x72, 0x6f, 0x6d, 0x65,
0x74, 0x65, 0x72, 0x3a, 0x6d, 0x69, 0x63, 0x72, 0x6f, 0x6d, 0x65, 0x74, 0x65, 0x72, 0x2d, 0x72, 0x65, 0x67, 0x69, 0x73,
0x74, 0x72, 0x79, 0x2d, 0x70, 0x72, 0x6f, 0x6d, 0x65, 0x74, 0x68, 0x65, 0x75, 0x73, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x6b, 0x6f, 0x74,
0x6c, 0x69, 0x6e, 0x28, 0x22, 0x73, 0x74, 0x64, 0x6c, 0x69, 0x62, 0x22, 0x29, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f,
0x2f, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x20, 0x54, 0x65, 0x73, 0x74, 0x20, 0x64, 0x65, 0x70, 0x65, 0x6e, 0x64,
0x65, 0x6e, 0x63, 0x69, 0x65, 0x73, 0x2e, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f,
0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x2f, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x74,
0x65, 0x73, 0x74, 0x49, 0x6d, 0x70, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x22, 0x6a,
0x66, 0x75, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x65, 0x3a, 0x70, 0x65, 0x2d, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2d,
0x74, 0x65, 0x73, 0x74, 0x22, 0x29, 0x0a, 0x7d, 0x0a, 0x0a, 0x76, 0x61, 0x6c, 0x20, 0x70, 0x61, 0x74, 0x63, 0x68, 0x4a,
0x61, 0x72, 0x20, 0x62, 0x79, 0x20, 0x74, 0x61, 0x73, 0x6b, 0x73, 0x2e, 0x72, 0x65, 0x67, 0x69, 0x73, 0x74, 0x65, 0x72,
0x69, 0x6e, 0x67, 0x28, 0x4a, 0x61, 0x72, 0x3a, 0x3a, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20,
0x20, 0x20, 0x61, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x69, 0x66, 0x69, 0x65, 0x72, 0x2e,
0x73, 0x65, 0x74, 0x28, 0x22, 0x70, 0x61, 0x74, 0x63, 0x68, 0x65, 0x64, 0x22, 0x29, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20,
0x76, 0x61, 0x6c, 0x20, 0x72, 0x75, 0x6e, 0x74, 0x69, 0x6d, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x70, 0x61, 0x74, 0x68,
0x20, 0x62, 0x79, 0x20, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x67,
0x65, 0x74, 0x74, 0x69, 0x6e, 0x67, 0x0a, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x6d, 0x61, 0x6e, 0x69, 0x66, 0x65, 0x73, 0x74,
0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65,
0x73, 0x28, 0x22, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x2d, 0x50, 0x61, 0x74, 0x68, 0x22, 0x20, 0x74, 0x6f, 0x20, 0x6f, 0x62,
0x6a, 0x65, 0x63, 0x74, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x70,
0x72, 0x69, 0x76, 0x61, 0x74, 0x65, 0x20, 0x76, 0x61, 0x6c, 0x20, 0x70, 0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x20, 0x3d,
0x20, 0x22, 0x66, 0x69, 0x6c, 0x65, 0x3a, 0x2f, 0x2b, 0x22, 0x2e, 0x74, 0x6f, 0x52, 0x65, 0x67, 0x65, 0x78, 0x28, 0x29,
0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x6f, 0x76, 0x65, 0x72, 0x72, 0x69, 0x64,
0x65, 0x20, 0x66, 0x75, 0x6e, 0x20, 0x74, 0x6f, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x28, 0x29, 0x3a, 0x20, 0x53, 0x74,
0x72, 0x69, 0x6e, 0x67, 0x20, 0x3d, 0x20, 0x72, 0x75, 0x6e, 0x74, 0x69, 0x6d, 0x65, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x70,
0x61, 0x74, 0x68, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x73, 0x2e, 0x6a, 0x6f, 0x69, 0x6e, 0x54, 0x6f, 0x53, 0x74, 0x72, 0x69,
0x6e, 0x67, 0x28, 0x22, 0x20, 0x22, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x69, 0x74, 0x2e, 0x74, 0x6f, 0x55, 0x52, 0x49, 0x28, 0x29, 0x2e, 0x74, 0x6f, 0x55,
0x52, 0x4c, 0x28, 0x29, 0x2e, 0x74, 0x6f, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x28, 0x29, 0x2e, 0x72, 0x65, 0x70, 0x6c,
0x61, 0x63, 0x65, 0x46, 0x69, 0x72, 0x73, 0x74, 0x28, 0x70, 0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x2c, 0x20, 0x22, 0x2f,
0x22, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x20, 0x20, 0x20,
0x20, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x7d, 0x0a, 0x0a, 0x74, 0x61, 0x73,
0x6b, 0x73, 0x2e, 0x6e, 0x61, 0x6d, 0x65, 0x64, 0x3c, 0x42, 0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x3e, 0x28, 0x22, 0x62,
0x6f, 0x6f, 0x74, 0x52, 0x75, 0x6e, 0x22, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x69, 0x66, 0x20, 0x28, 0x4f,
0x73, 0x2e, 0x69, 0x73, 0x46, 0x61, 0x6d, 0x69, 0x6c, 0x79, 0x28, 0x4f, 0x73, 0x2e, 0x46, 0x41, 0x4d, 0x49, 0x4c, 0x59,
0x5f, 0x57, 0x49, 0x4e, 0x44, 0x4f, 0x57, 0x53, 0x29, 0x29, 0x20, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
0x20, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x70, 0x61, 0x74, 0x68, 0x20, 0x3d, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x73, 0x28, 0x73,
0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x74, 0x73, 0x2e, 0x6e, 0x61, 0x6d, 0x65, 0x64, 0x28, 0x22, 0x6d, 0x61, 0x69,
0x6e, 0x22, 0x29, 0x2e, 0x6d, 0x61, 0x70, 0x20, 0x7b, 0x20, 0x69, 0x74, 0x2e, 0x6f, 0x75, 0x74, 0x70, 0x75, 0x74, 0x20,
0x7d, 0x2c, 0x20, 0x70, 0x61, 0x74, 0x63, 0x68, 0x4a, 0x61, 0x72, 0x29, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x7d, 0x0a, 0x0a,
0x20, 0x20, 0x20, 0x20, 0x2f, 0x2f, 0x20, 0xd0,
}

View File

@ -19,7 +19,6 @@ import (
charsetModule "code.gitea.io/gitea/modules/charset"
"code.gitea.io/gitea/modules/container"
"code.gitea.io/gitea/modules/httpcache"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/typesniffer"
"code.gitea.io/gitea/modules/util"
@ -109,11 +108,7 @@ func setServeHeadersByFile(r *http.Request, w http.ResponseWriter, mineBuf []byt
}
if isPlain {
charset, err := charsetModule.DetectEncoding(mineBuf)
if err != nil {
log.Error("Detect raw file %s charset failed: %v, using by default utf-8", opts.Filename, err)
charset = "utf-8"
}
charset, _ := charsetModule.DetectEncoding(mineBuf)
opts.ContentTypeCharset = strings.ToLower(charset)
}

View File

@ -203,7 +203,7 @@ func (b *Indexer) addUpdate(ctx context.Context, batchWriter git.WriteCloserErro
RepoID: repo.ID,
CommitID: commitSha,
Filename: update.Filename,
Content: string(charset.ToUTF8DropErrors(fileContents, charset.ConvertOpts{})),
Content: string(charset.ToUTF8DropErrors(fileContents)),
Language: analyze.GetCodeLanguage(update.Filename, fileContents),
UpdatedAt: time.Now().UTC(),
})

View File

@ -191,7 +191,7 @@ func (b *Indexer) addUpdate(ctx context.Context, batchWriter git.WriteCloserErro
Doc(map[string]any{
"repo_id": repo.ID,
"filename": update.Filename,
"content": string(charset.ToUTF8DropErrors(fileContents, charset.ConvertOpts{})),
"content": string(charset.ToUTF8DropErrors(fileContents)),
"commit_id": sha,
"language": analyze.GetCodeLanguage(update.Filename, fileContents),
"updated_at": timeutil.TimeStampNow(),

View File

@ -240,4 +240,5 @@ func PanicInDevOrTesting(msg string, a ...any) {
if !IsProd || IsInTesting {
panic(fmt.Sprintf(msg, a...))
}
log.Error(msg, a...)
}

View File

@ -0,0 +1,49 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package structs
// ActionsPermissions represents Actions token permissions for a repository
// swagger:model
type ActionsPermissions struct {
PermissionMode int `json:"permission_mode"`
ActionsRead bool `json:"actions_read"`
ActionsWrite bool `json:"actions_write"`
ContentsRead bool `json:"contents_read"`
ContentsWrite bool `json:"contents_write"`
IssuesRead bool `json:"issues_read"`
IssuesWrite bool `json:"issues_write"`
PackagesRead bool `json:"packages_read"`
PackagesWrite bool `json:"packages_write"`
PullRequestsRead bool `json:"pull_requests_read"`
PullRequestsWrite bool `json:"pull_requests_write"`
MetadataRead bool `json:"metadata_read"`
}
// OrgActionsPermissions represents organization-level Actions token permissions
// swagger:model
type OrgActionsPermissions struct {
PermissionMode int `json:"permission_mode"`
AllowRepoOverride bool `json:"allow_repo_override"`
ActionsRead bool `json:"actions_read"`
ActionsWrite bool `json:"actions_write"`
ContentsRead bool `json:"contents_read"`
ContentsWrite bool `json:"contents_write"`
IssuesRead bool `json:"issues_read"`
IssuesWrite bool `json:"issues_write"`
PackagesRead bool `json:"packages_read"`
PackagesWrite bool `json:"packages_write"`
PullRequestsRead bool `json:"pull_requests_read"`
PullRequestsWrite bool `json:"pull_requests_write"`
MetadataRead bool `json:"metadata_read"`
}
// CrossRepoAccessRule represents a cross-repository access rule
// swagger:model
type CrossRepoAccessRule struct {
ID int64 `json:"id"`
OrgID int64 `json:"org_id"`
SourceRepoID int64 `json:"source_repo_id"`
TargetRepoID int64 `json:"target_repo_id"`
AccessLevel int `json:"access_level"`
}

View File

@ -1271,6 +1271,11 @@ func Routes() *web.Router {
})
}, reqToken(), reqAdmin())
m.Group("/actions", func() {
m.Group("/permissions", func() {
m.Get("", reqAdmin(), repo.GetActionsPermissions)
m.Put("", reqAdmin(), repo.UpdateActionsPermissions)
}, reqToken())
m.Get("/tasks", repo.ListActionTasks)
m.Group("/runs", func() {
m.Group("/{run}", func() {
@ -1619,6 +1624,18 @@ func Routes() *web.Router {
m.Post("/orgs", tokenRequiresScopes(auth_model.AccessTokenScopeCategoryOrganization), reqToken(), bind(api.CreateOrgOption{}), org.Create)
m.Get("/orgs", org.GetAll, tokenRequiresScopes(auth_model.AccessTokenScopeCategoryOrganization))
m.Group("/orgs/{org}", func() {
m.Group("/settings/actions", func() {
m.Group("/permissions", func() {
m.Get("", reqOrgOwnership(), org.GetActionsPermissions)
m.Put("", reqOrgOwnership(), org.UpdateActionsPermissions)
})
m.Group("/cross-repo-access", func() {
m.Get("", reqOrgOwnership(), org.ListCrossRepoAccess)
m.Post("", reqOrgOwnership(), org.AddCrossRepoAccess)
m.Delete("/{id}", reqOrgOwnership(), org.DeleteCrossRepoAccess)
})
}, reqToken(), context.OrgAssignment(context.OrgAssignmentOptions{}))
m.Combo("").Get(org.Get).
Patch(reqToken(), reqOrgOwnership(), bind(api.EditOrgOption{}), org.Edit).
Delete(reqToken(), reqOrgOwnership(), org.Delete)

View File

@ -0,0 +1,301 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package org
import (
"net/http"
actions_model "code.gitea.io/gitea/models/actions"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/modules/web"
"code.gitea.io/gitea/services/context"
)
// GetActionsPermissions returns the Actions token permissions for an organization
func GetActionsPermissions(ctx *context.APIContext) {
// swagger:operation GET /orgs/{org}/settings/actions/permissions organization orgGetActionsPermissions
// ---
// summary: Get organization Actions token permissions
// produces:
// - application/json
// parameters:
// - name: org
// in: path
// description: name of the organization
// type: string
// required: true
// responses:
// "200":
// "$ref": "#/responses/OrgActionsPermissionsResponse"
// "404":
// "$ref": "#/responses/notFound"
// Organization settings are more sensitive than repo settings because they
// affect ALL repositories in the org. We should be extra careful here.
// Only org owners should be able to modify these settings.
// This is enforced by the reqOrgOwnership middleware.
perms, err := actions_model.GetOrgActionPermissions(ctx, ctx.Org.Organization.ID)
if err != nil {
ctx.APIErrorInternal(err)
return
}
// Return default if no custom config exists
// Organizations default to restricted mode for maximum security
// Individual repos can be given more permissions if needed
if perms == nil {
perms = &actions_model.ActionOrgPermission{
OrgID: ctx.Org.Organization.ID,
PermissionMode: actions_model.PermissionModeRestricted,
AllowRepoOverride: true, // Allow repos to configure their own settings
ContentsRead: true,
MetadataRead: true,
}
}
ctx.JSON(http.StatusOK, convertToAPIOrgPermissions(perms))
}
// UpdateActionsPermissions updates the Actions token permissions for an organization
func UpdateActionsPermissions(ctx *context.APIContext) {
// swagger:operation PUT /orgs/{org}/settings/actions/permissions organization orgUpdateActionsPermissions
// ---
// summary: Update organization Actions token permissions
// consumes:
// - application/json
// produces:
// - application/json
// parameters:
// - name: org
// in: path
// description: name of the organization
// type: string
// required: true
// - name: body
// in: body
// schema:
// "$ref": "#/definitions/OrgActionsPermissions"
// responses:
// "200":
// "$ref": "#/responses/OrgActionsPermissionsResponse"
// "403":
// "$ref": "#/responses/forbidden"
// Organization settings are more sensitive than repo settings because they
// affect ALL repositories in the org. We should be extra careful here.
// Only org owners should be able to modify these settings.
// This is enforced by the reqOrgOwnership middleware.
form := web.GetForm(ctx).(*api.OrgActionsPermissions)
// Validate permission mode
if form.PermissionMode < 0 || form.PermissionMode > 2 {
ctx.APIError(http.StatusUnprocessableEntity, "Permission mode must be 0 (restricted), 1 (permissive), or 2 (custom)")
return
}
// Important security consideration:
// If AllowRepoOverride is false, ALL repos in this org MUST use org settings.
// This is useful for security-conscious organizations that want centralized control.
// However, it's a big change, so we should log this action for audit purposes.
// TODO: Add audit logging when this feature is used
perm := &actions_model.ActionOrgPermission{
OrgID: ctx.Org.Organization.ID,
PermissionMode: actions_model.PermissionMode(form.PermissionMode),
AllowRepoOverride: form.AllowRepoOverride,
ActionsRead: form.ActionsRead,
ActionsWrite: form.ActionsWrite,
ContentsRead: form.ContentsRead,
ContentsWrite: form.ContentsWrite,
IssuesRead: form.IssuesRead,
IssuesWrite: form.IssuesWrite,
PackagesRead: form.PackagesRead,
PackagesWrite: form.PackagesWrite,
PullRequestsRead: form.PullRequestsRead,
PullRequestsWrite: form.PullRequestsWrite,
MetadataRead: true, // Always true
}
if err := actions_model.CreateOrUpdateOrgPermissions(ctx, perm); err != nil {
ctx.APIErrorInternal(err)
return
}
// If AllowRepoOverride is false, we might want to update all repo permissions
// to match org settings. But that's a big operation, so let's do it lazily
// when permissions are actually checked, rather than updating all repos here.
// This is more performant and avoids potential race conditions.
ctx.JSON(http.StatusOK, convertToAPIOrgPermissions(perm))
}
// ListCrossRepoAccess lists all cross-repository access rules for an organization
func ListCrossRepoAccess(ctx *context.APIContext) {
// swagger:operation GET /orgs/{org}/settings/actions/cross-repo-access organization orgListCrossRepoAccess
// ---
// summary: List cross-repository access rules
// produces:
// - application/json
// parameters:
// - name: org
// in: path
// description: name of the organization
// type: string
// required: true
// responses:
// "200":
// "$ref": "#/responses/CrossRepoAccessList"
// This is a critical security feature - cross-repo access allows one repo's
// Actions to access another repo's code/resources. We need to be very careful
// about how we implement this. See the discussion:
// https://github.com/go-gitea/gitea/issues/24635
// Permission check handled by reqOrgOwnership middleware
rules, err := actions_model.ListCrossRepoAccessRules(ctx, ctx.Org.Organization.ID)
if err != nil {
ctx.APIErrorInternal(err)
return
}
apiRules := make([]*api.CrossRepoAccessRule, len(rules))
for i, rule := range rules {
apiRules[i] = convertToCrossRepoAccessRule(rule)
}
ctx.JSON(http.StatusOK, apiRules)
}
// AddCrossRepoAccess adds a new cross-repository access rule
func AddCrossRepoAccess(ctx *context.APIContext) {
// swagger:operation POST /orgs/{org}/settings/actions/cross-repo-access organization orgAddCrossRepoAccess
// ---
// summary: Add cross-repository access rule
// consumes:
// - application/json
// produces:
// - application/json
// parameters:
// - name: org
// in: path
// description: name of the organization
// type: string
// required: true
// - name: body
// in: body
// schema:
// "$ref": "#/definitions/CrossRepoAccessRule"
// responses:
// "201":
// "$ref": "#/responses/CrossRepoAccessRule"
// "403":
// "$ref": "#/responses/forbidden"
// Permission check handled by reqOrgOwnership middleware
form := web.GetForm(ctx).(*api.CrossRepoAccessRule)
// Validation: source and target repos must both belong to this org
// We don't want to allow cross-organization access - that would be a
// security nightmare and makes audit trails very complex.
// TODO: Verify both repos belong to this org
// Validation: Access level must be valid (0=none, 1=read, 2=write)
if form.AccessLevel < 0 || form.AccessLevel > 2 {
ctx.APIError(http.StatusUnprocessableEntity, "Access level must be 0 (none), 1 (read), or 2 (write)")
return
}
rule := &actions_model.ActionCrossRepoAccess{
OrgID: ctx.Org.Organization.ID,
SourceRepoID: form.SourceRepoID,
TargetRepoID: form.TargetRepoID,
AccessLevel: form.AccessLevel,
}
if err := actions_model.CreateCrossRepoAccess(ctx, rule); err != nil {
ctx.APIErrorInternal(err)
return
}
ctx.JSON(http.StatusCreated, convertToCrossRepoAccessRule(rule))
}
// DeleteCrossRepoAccess removes a cross-repository access rule
func DeleteCrossRepoAccess(ctx *context.APIContext) {
// swagger:operation DELETE /orgs/{org}/settings/actions/cross-repo-access/{id} organization orgDeleteCrossRepoAccess
// ---
// summary: Delete cross-repository access rule
// parameters:
// - name: org
// in: path
// description: name of the organization
// type: string
// required: true
// - name: id
// in: path
// description: ID of the rule to delete
// type: integer
// required: true
// responses:
// "204":
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// Permission check handled by reqOrgOwnership middleware
ruleID := ctx.PathParamInt64("id")
// Security check: Verify the rule belongs to this org before deleting
// We don't want one org to be able to delete another org's rules
rule, err := actions_model.GetCrossRepoAccessByID(ctx, ruleID)
if err != nil {
ctx.APIError(http.StatusNotFound, "Cross-repo access rule not found")
return
}
if rule.OrgID != ctx.Org.Organization.ID {
ctx.APIError(http.StatusForbidden, "This rule belongs to a different organization")
return
}
if err := actions_model.DeleteCrossRepoAccess(ctx, ruleID); err != nil {
ctx.APIErrorInternal(err)
return
}
ctx.Status(http.StatusNoContent)
}
// Helper functions
func convertToAPIOrgPermissions(perm *actions_model.ActionOrgPermission) *api.OrgActionsPermissions {
return &api.OrgActionsPermissions{
PermissionMode: int(perm.PermissionMode),
AllowRepoOverride: perm.AllowRepoOverride,
ActionsRead: perm.ActionsRead,
ActionsWrite: perm.ActionsWrite,
ContentsRead: perm.ContentsRead,
ContentsWrite: perm.ContentsWrite,
IssuesRead: perm.IssuesRead,
IssuesWrite: perm.IssuesWrite,
PackagesRead: perm.PackagesRead,
PackagesWrite: perm.PackagesWrite,
PullRequestsRead: perm.PullRequestsRead,
PullRequestsWrite: perm.PullRequestsWrite,
MetadataRead: perm.MetadataRead,
}
}
func convertToCrossRepoAccessRule(rule *actions_model.ActionCrossRepoAccess) *api.CrossRepoAccessRule {
return &api.CrossRepoAccessRule{
ID: rule.ID,
OrgID: rule.OrgID,
SourceRepoID: rule.SourceRepoID,
TargetRepoID: rule.TargetRepoID,
AccessLevel: rule.AccessLevel,
}
}

View File

@ -0,0 +1,201 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package repo
import (
"net/http"
actions_model "code.gitea.io/gitea/models/actions"
api "code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/modules/web"
"code.gitea.io/gitea/services/context"
)
// swagger:operation GET /repos/{owner}/{repo}/settings/actions/permissions repository repoGetActionsPermissions
// ---
// summary: Get repository Actions token permissions
// produces:
// - application/json
// parameters:
// - name: owner
// in: path
// description: owner of the repo
// type: string
// required: true
// - name: repo
// in: path
// description: name of the repo
// type: string
// required: true
// responses:
// "200":
// "$ref": "#/responses/ActionsPermissionsResponse"
// "404":
// "$ref": "#/responses/notFound"
// GetActionsPermissions returns the Actions token permissions for a repository
func GetActionsPermissions(ctx *context.APIContext) {
// Check if user has admin access to this repo
// NOTE: Only repo admins and owners should be able to view/modify permission settings
// This is important for security - we don't want regular contributors
// to be able to grant themselves elevated permissions via Actions
// This is enforced by the reqAdmin middleware.
perms, err := actions_model.GetRepoActionPermissions(ctx, ctx.Repo.Repository.ID)
if err != nil {
ctx.APIErrorInternal(err)
return
}
// If no custom permissions are set, return the default (restricted mode)
// This is intentional - we want a secure default that requires explicit opt-in
// to more permissive settings. See: https://github.com/go-gitea/gitea/issues/24635
if perms == nil {
perms = &actions_model.ActionTokenPermission{
RepoID: ctx.Repo.Repository.ID,
PermissionMode: actions_model.PermissionModeRestricted,
// Default restricted permissions - only read contents and metadata
ContentsRead: true,
MetadataRead: true,
}
}
ctx.JSON(http.StatusOK, convertToAPIPermissions(perms))
}
// swagger:operation PUT /repos/{owner}/{repo}/settings/actions/permissions repository repoUpdateActionsPermissions
// ---
// summary: Update repository Actions token permissions
// consumes:
// - application/json
// produces:
// - application/json
// parameters:
// - name: owner
// in: path
// description: owner of the repo
// type: string
// required: true
// - name: repo
// in: path
// description: name of the repo
// type: string
// required: true
// - name: body
// in: body
// schema:
// "$ref": "#/definitions/ActionsPermissions"
// responses:
// "200":
// "$ref": "#/responses/ActionsPermissionsResponse"
// "403":
// "$ref": "#/responses/forbidden"
// "422":
// "$ref": "#/responses/validationError"
// UpdateActionsPermissions updates the Actions token permissions for a repository
func UpdateActionsPermissions(ctx *context.APIContext) {
// Only repo admins and owners should be able to modify these settings.
// This is enforced by the reqAdmin middleware.
form := web.GetForm(ctx).(*api.ActionsPermissions)
// Validate permission mode
if form.PermissionMode < 0 || form.PermissionMode > 2 {
ctx.APIError(http.StatusUnprocessableEntity, "Permission mode must be 0 (restricted), 1 (permissive), or 2 (custom)")
return
}
// TODO: Check if org-level permissions exist and validate against them
// For now, we'll implement basic validation, but we should enhance this
// to ensure repo settings don't exceed org caps. This is important for
// multi-repository organizations where admins want centralized control.
// See wolfogre's comment: https://github.com/go-gitea/gitea/pull/24554#issuecomment-1537040811
perm := &actions_model.ActionTokenPermission{
RepoID: ctx.Repo.Repository.ID,
PermissionMode: actions_model.PermissionMode(form.PermissionMode),
ActionsRead: form.ActionsRead,
ActionsWrite: form.ActionsWrite,
ContentsRead: form.ContentsRead,
ContentsWrite: form.ContentsWrite,
IssuesRead: form.IssuesRead,
IssuesWrite: form.IssuesWrite,
PackagesRead: form.PackagesRead,
PackagesWrite: form.PackagesWrite,
PullRequestsRead: form.PullRequestsRead,
PullRequestsWrite: form.PullRequestsWrite,
MetadataRead: true, // Always true - needed for basic operations
}
if err := actions_model.CreateOrUpdateRepoPermissions(ctx, perm); err != nil {
ctx.APIErrorInternal(err)
return
}
ctx.JSON(http.StatusOK, convertToAPIPermissions(perm))
}
// ResetActionsPermissions resets permissions to default (restricted mode)
func ResetActionsPermissions(ctx *context.APIContext) {
// swagger:operation DELETE /repos/{owner}/{repo}/settings/actions/permissions repository repoResetActionsPermissions
// ---
// summary: Reset repository Actions permissions to default
// produces:
// - application/json
// parameters:
// - name: owner
// in: path
// description: owner of the repo
// type: string
// required: true
// - name: repo
// in: path
// description: name of the repo
// type: string
// required: true
// responses:
// "204":
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// Only repo admins and owners should be able to modify these settings.
// This is enforced by the reqAdmin middleware.
// Create default restricted permissions
// This is a "safe reset" - puts the repo back to secure defaults
defaultPerm := &actions_model.ActionTokenPermission{
RepoID: ctx.Repo.Repository.ID,
PermissionMode: actions_model.PermissionModeRestricted,
ContentsRead: true,
MetadataRead: true,
}
if err := actions_model.CreateOrUpdateRepoPermissions(ctx, defaultPerm); err != nil {
ctx.APIErrorInternal(err)
return
}
ctx.Status(http.StatusNoContent)
}
// convertToAPIPermissions converts model to API response format
// This helper keeps our internal model separate from the API contract
func convertToAPIPermissions(perm *actions_model.ActionTokenPermission) *api.ActionsPermissions {
return &api.ActionsPermissions{
PermissionMode: int(perm.PermissionMode),
ActionsRead: perm.ActionsRead,
ActionsWrite: perm.ActionsWrite,
ContentsRead: perm.ContentsRead,
ContentsWrite: perm.ContentsWrite,
IssuesRead: perm.IssuesRead,
IssuesWrite: perm.IssuesWrite,
PackagesRead: perm.PackagesRead,
PackagesWrite: perm.PackagesWrite,
PullRequestsRead: perm.PullRequestsRead,
PullRequestsWrite: perm.PullRequestsWrite,
MetadataRead: perm.MetadataRead,
}
}

View File

@ -317,11 +317,7 @@ func EditFile(ctx *context.Context) {
ctx.ServerError("ReadAll", err)
return
}
if content, err := charset.ToUTF8(buf, charset.ConvertOpts{KeepBOM: true}); err != nil {
ctx.Data["FileContent"] = string(buf)
} else {
ctx.Data["FileContent"] = content
}
ctx.Data["FileContent"] = string(charset.ToUTF8(buf, charset.ConvertOpts{KeepBOM: true, ErrorReturnOrigin: true}))
}
}

View File

@ -835,11 +835,11 @@ parsingLoop:
if buffer.Len() == 0 {
continue
}
charsetLabel, err := charset.DetectEncoding(buffer.Bytes())
if charsetLabel != "UTF-8" && err == nil {
encoding, _ := stdcharset.Lookup(charsetLabel)
if encoding != nil {
diffLineTypeDecoders[lineType] = encoding.NewDecoder()
charsetLabel, _ := charset.DetectEncoding(buffer.Bytes())
if charsetLabel != "UTF-8" {
charsetEncoding, _ := stdcharset.Lookup(charsetLabel)
if charsetEncoding != nil {
diffLineTypeDecoders[lineType] = charsetEncoding.NewDecoder()
}
}
}
@ -1325,10 +1325,10 @@ func GetDiffForRender(ctx context.Context, repoLink string, gitRepo *git.Reposit
shouldFullFileHighlight := !setting.Git.DisableDiffHighlight && attrDiff.Value() == ""
if shouldFullFileHighlight {
if limitedContent.LeftContent != nil && limitedContent.LeftContent.buf.Len() < MaxDiffHighlightEntireFileSize {
diffFile.highlightedLeftLines = highlightCodeLines(diffFile, true /* left */, limitedContent.LeftContent.buf.String())
diffFile.highlightedLeftLines = highlightCodeLines(diffFile, true /* left */, limitedContent.LeftContent.buf.Bytes())
}
if limitedContent.RightContent != nil && limitedContent.RightContent.buf.Len() < MaxDiffHighlightEntireFileSize {
diffFile.highlightedRightLines = highlightCodeLines(diffFile, false /* right */, limitedContent.RightContent.buf.String())
diffFile.highlightedRightLines = highlightCodeLines(diffFile, false /* right */, limitedContent.RightContent.buf.Bytes())
}
}
}
@ -1336,9 +1336,34 @@ func GetDiffForRender(ctx context.Context, repoLink string, gitRepo *git.Reposit
return diff, nil
}
func highlightCodeLines(diffFile *DiffFile, isLeft bool, content string) map[int]template.HTML {
func splitHighlightLines(buf []byte) (ret [][]byte) {
lineCount := bytes.Count(buf, []byte("\n")) + 1
ret = make([][]byte, 0, lineCount)
nlTagClose := []byte("\n</")
for {
pos := bytes.IndexByte(buf, '\n')
if pos == -1 {
ret = append(ret, buf)
return ret
}
// Chroma highlighting output sometimes have "</span>" right after \n, sometimes before.
// * "<span>text\n</span>"
// * "<span>text</span>\n"
if bytes.HasPrefix(buf[pos:], nlTagClose) {
pos1 := bytes.IndexByte(buf[pos:], '>')
if pos1 != -1 {
pos += pos1
}
}
ret = append(ret, buf[:pos+1])
buf = buf[pos+1:]
}
}
func highlightCodeLines(diffFile *DiffFile, isLeft bool, rawContent []byte) map[int]template.HTML {
content := util.UnsafeBytesToString(charset.ToUTF8(rawContent, charset.ConvertOpts{}))
highlightedNewContent, _ := highlight.Code(diffFile.Name, diffFile.Language, content)
splitLines := strings.Split(string(highlightedNewContent), "\n")
splitLines := splitHighlightLines([]byte(highlightedNewContent))
lines := make(map[int]template.HTML, len(splitLines))
// only save the highlighted lines we need, but not the whole file, to save memory
for _, sec := range diffFile.Sections {

View File

@ -5,6 +5,7 @@
package gitdiff
import (
"html/template"
"strconv"
"strings"
"testing"
@ -1106,3 +1107,41 @@ func TestDiffLine_GetExpandDirection(t *testing.T) {
assert.Equal(t, c.direction, c.diffLine.GetExpandDirection(), "case %s expected direction: %s", c.name, c.direction)
}
}
func TestHighlightCodeLines(t *testing.T) {
t.Run("CharsetDetecting", func(t *testing.T) {
diffFile := &DiffFile{
Name: "a.c",
Language: "c",
Sections: []*DiffSection{
{
Lines: []*DiffLine{{LeftIdx: 1}},
},
},
}
ret := highlightCodeLines(diffFile, true, []byte("// abc\xcc def\xcd")) // ISO-8859-1 bytes
assert.Equal(t, "<span class=\"c1\">// abcÌ defÍ\n</span>", string(ret[0]))
})
t.Run("LeftLines", func(t *testing.T) {
diffFile := &DiffFile{
Name: "a.c",
Language: "c",
Sections: []*DiffSection{
{
Lines: []*DiffLine{
{LeftIdx: 1},
{LeftIdx: 2},
{LeftIdx: 3},
},
},
},
}
const nl = "\n"
ret := highlightCodeLines(diffFile, true, []byte("a\nb\n"))
assert.Equal(t, map[int]template.HTML{
0: `<span class="n">a</span>` + nl,
1: `<span class="n">b</span>`,
}, ret)
})
}

View File

@ -25,12 +25,12 @@ func TestDiffWithHighlight(t *testing.T) {
t.Run("CleanUp", func(t *testing.T) {
hcd := newHighlightCodeDiff()
codeA := template.HTML(`<span class="cm>this is a comment</span>`)
codeB := template.HTML(`<span class="cm>this is updated comment</span>`)
codeA := template.HTML(`<span class="cm">this is a comment</span>`)
codeB := template.HTML(`<span class="cm">this is updated comment</span>`)
outDel := hcd.diffLineWithHighlight(DiffLineDel, codeA, codeB)
assert.Equal(t, `<span class="cm>this is <span class="removed-code">a</span> comment</span>`, string(outDel))
assert.Equal(t, `<span class="cm">this is <span class="removed-code">a</span> comment</span>`, string(outDel))
outAdd := hcd.diffLineWithHighlight(DiffLineAdd, codeA, codeB)
assert.Equal(t, `<span class="cm>this is <span class="added-code">updated</span> comment</span>`, string(outAdd))
assert.Equal(t, `<span class="cm">this is <span class="added-code">updated</span> comment</span>`, string(outAdd))
})
t.Run("OpenCloseTags", func(t *testing.T) {

View File

@ -0,0 +1,242 @@
{{template "base/head" .}}
<div class="page-content repository settings options">
{{template "repo/header" .}}
<div class="ui container">
<div class="ui grid">
{{template "repo/settings/navbar" .}}
<div class="twelve wide column content">
{{template "base/alert" .}}
<h4 class="ui top attached header">
{{.locale.Tr "repo.settings.actions.permissions.title"}}
</h4>
<div class="ui attached segment">
<p class="help">
{{.locale.Tr "repo.settings.actions.permissions.desc"}}
<!-- TODO: Add link to documentation once it's written -->
<!-- Need to explain this feature clearly for users -->
</p>
<form class="ui form" method="post" action="{{.Link}}">
{{.CsrfTokenHtml}}
<!-- Permission Mode Selector -->
<div class="field">
<label>{{.locale.Tr "repo.settings.actions.permissions.mode"}}</label>
<div class="ui selection dropdown">
<input type="hidden" name="permission_mode" value="{{.PermissionMode}}">
<i class="dropdown icon"></i>
<div class="default text">Select permission mode</div>
<div class="menu">
<!-- Restricted mode - recommended for most users -->
<div class="item" data-value="0" data-text="Restricted (Recommended)">
<div class="header">🔒 Restricted (Recommended)</div>
<div class="description">
Minimal permissions. Actions can only read code. Secure default.
</div>
</div>
<!-- Permissive mode - for trusted repos -->
<div class="item" data-value="1" data-text="Permissive">
<div class="header">🔓 Permissive</div>
<div class="description">
Broad permissions. Actions can read/write most resources. For trusted environments only.
</div>
</div>
<!-- Custom mode - for advanced users -->
<div class="item" data-value="2" data-text="Custom">
<div class="header">⚙️ Custom</div>
<div class="description">
Fine-grained control. Configure each permission individually.
</div>
</div>
</div>
</div>
</div>
<!-- Custom permissions - only shown when mode is Custom -->
<!-- Note: We could use Vue.js here for reactivity, but keeping it simple with vanilla JS -->
<!-- If this gets more complex, consider refactoring to use Vue component -->
<div id="custom-permissions" class="{{if ne .PermissionMode 2}}hide{{end}}">
<div class="ui divider"></div>
<h5>Individual Permissions</h5>
{{/* Actions Permission */}}
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="actions_read" id="actions_read" {{if .ActionsRead}}checked{{end}}>
<label for="actions_read">
<strong>Actions (Read)</strong> - View workflow runs and logs
</label>
</div>
</div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="actions_write" id="actions_write" {{if .ActionsWrite}}checked{{end}}>
<label for="actions_write">
<strong>Actions (Write)</strong> - Cancel or re-run workflows
</label>
</div>
</div>
{{/* Contents Permission */}}
<div class="ui divider"></div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="contents_read" id="contents_read" {{if .ContentsRead}}checked{{end}}>
<label for="contents_read">
<strong>Contents (Read)</strong> - Clone and read repository code
<span class="text grey">(Recommended: Keep enabled)</span>
</label>
</div>
</div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="contents_write" id="contents_write" {{if .ContentsWrite}}checked{{end}}>
<label for="contents_write">
<strong>Contents (Write)</strong> - Push commits and create branches
<span class="text red">(Warning: High risk for fork PRs)</span>
</label>
</div>
</div>
{{/* Packages Permission */}}
<div class="ui divider"></div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="packages_read" id="packages_read" {{if .PackagesRead}}checked{{end}}>
<label for="packages_read">
<strong>Packages (Read)</strong> - Pull packages from registry
</label>
</div>
</div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="packages_write" id="packages_write" {{if .PackagesWrite}}checked{{end}}>
<label for="packages_write">
<strong>Packages (Write)</strong> - Publish and update packages
<!-- Note: Requires package-repository linking (see org settings) -->
</label>
</div>
</div>
{{/* Issues Permission */}}
<div class="ui divider"></div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="issues_read" id="issues_read" {{if .IssuesRead}}checked{{end}}>
<label for="issues_read">
<strong>Issues (Read)</strong> - View issues
</label>
</div>
</div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="issues_write" id="issues_write" {{if .IssuesWrite}}checked{{end}}>
<label for="issues_write">
<strong>Issues (Write)</strong> - Create, comment, and close issues
</label>
</div>
</div>
{{/* Pull Requests Permission */}}
<div class="ui divider"></div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="pull_requests_read" id="pull_requests_read" {{if .PullRequestsRead}}checked{{end}}>
<label for="pull_requests_read">
<strong>Pull Requests (Read)</strong> - View pull requests
</label>
</div>
</div>
<div class="field">
<div class="ui toggle checkbox">
<input type="checkbox" name="pull_requests_write" id="pull_requests_write" {{if .PullRequestsWrite}}checked{{end}}>
<label for="pull_requests_write">
<strong>Pull Requests (Write)</strong> - Create and merge pull requests
</label>
</div>
</div>
</div>
<!-- Warning Message for fork PRs -->
<!-- This is important - users need to understand that fork PRs are always restricted -->
<div class="ui warning message">
<div class="header">
<i class="shield icon"></i>
Security Notice: Fork Pull Requests
</div>
<p>
For security reasons, workflows triggered by pull requests from forked repositories
are <strong>always restricted</strong> to read-only access, regardless of the settings above.
This prevents malicious forks from accessing secrets or modifying your repository.
</p>
<!-- Reference the security discussion that led to this decision -->
<!-- https://github.com/go-gitea/gitea/pull/24554#issuecomment-1537040811 -->
</div>
<!-- Organization Cap Notice (if applicable) -->
{{if .OrgID}}
{{if .OrgHasRestrictions}}
<div class="ui info message">
<div class="header">
<i class="building icon"></i>
Organization Restrictions Apply
</div>
<p>
This repository belongs to an organization with permission restrictions.
The settings above cannot exceed the organization's maximum permissions.
Contact your organization admin to grant additional permissions.
</p>
</div>
{{end}}
{{end}}
<!-- Submit Buttons -->
<div class="field">
<button class="ui green button" type="submit">
{{.locale.Tr "repo.settings.actions.permissions.save"}}
</button>
<a class="ui button" href="{{.Link}}">
{{.locale.Tr "repo.settings.cancel"}}
</a>
</div>
</form>
</div>
</div>
</div>
</div>
</div>
<!-- JavaScript for UI interactions -->
<script>
// Show/hide custom permissions based on mode selection
// TODO: Could move this to a separate JS file if it gets more complex
$(document).ready(function() {
// Drop down initialization
$('.ui.dropdown').dropdown({
onChange: function(value) {
// Show custom options only when Custom mode is selected
if (value === '2') {
$('#custom-permissions').removeClass('hide');
} else {
$('#custom-permissions').addClass('hide');
}
}
});
// Warning when enabling write permissions
// Helps prevent accidental security issues
$('#contents_write, #packages_write').change(function() {
if ($(this).is(':checked')) {
// Maybe add a confirmation dialog here?
// For now, just the inline warning text is probably enough
console.log('Write permission enabled - ensure this is intentional');
}
});
});
</script>
{{template "base/footer" .}}

View File

@ -3640,6 +3640,168 @@
}
}
},
"/orgs/{org}/settings/actions/cross-repo-access": {
"get": {
"produces": [
"application/json"
],
"tags": [
"organization"
],
"summary": "List cross-repository access rules",
"operationId": "orgListCrossRepoAccess",
"parameters": [
{
"type": "string",
"description": "name of the organization",
"name": "org",
"in": "path",
"required": true
}
],
"responses": {
"200": {
"$ref": "#/responses/CrossRepoAccessList"
}
}
},
"post": {
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"organization"
],
"summary": "Add cross-repository access rule",
"operationId": "orgAddCrossRepoAccess",
"parameters": [
{
"type": "string",
"description": "name of the organization",
"name": "org",
"in": "path",
"required": true
},
{
"name": "body",
"in": "body",
"schema": {
"$ref": "#/definitions/CrossRepoAccessRule"
}
}
],
"responses": {
"201": {
"$ref": "#/responses/CrossRepoAccessRule"
},
"403": {
"$ref": "#/responses/forbidden"
}
}
}
},
"/orgs/{org}/settings/actions/cross-repo-access/{id}": {
"delete": {
"tags": [
"organization"
],
"summary": "Delete cross-repository access rule",
"operationId": "orgDeleteCrossRepoAccess",
"parameters": [
{
"type": "string",
"description": "name of the organization",
"name": "org",
"in": "path",
"required": true
},
{
"type": "integer",
"description": "ID of the rule to delete",
"name": "id",
"in": "path",
"required": true
}
],
"responses": {
"204": {
"$ref": "#/responses/empty"
},
"403": {
"$ref": "#/responses/forbidden"
}
}
}
},
"/orgs/{org}/settings/actions/permissions": {
"get": {
"produces": [
"application/json"
],
"tags": [
"organization"
],
"summary": "Get organization Actions token permissions",
"operationId": "orgGetActionsPermissions",
"parameters": [
{
"type": "string",
"description": "name of the organization",
"name": "org",
"in": "path",
"required": true
}
],
"responses": {
"200": {
"$ref": "#/responses/OrgActionsPermissionsResponse"
},
"404": {
"$ref": "#/responses/notFound"
}
}
},
"put": {
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"organization"
],
"summary": "Update organization Actions token permissions",
"operationId": "orgUpdateActionsPermissions",
"parameters": [
{
"type": "string",
"description": "name of the organization",
"name": "org",
"in": "path",
"required": true
},
{
"name": "body",
"in": "body",
"schema": {
"$ref": "#/definitions/OrgActionsPermissions"
}
}
],
"responses": {
"200": {
"$ref": "#/responses/OrgActionsPermissionsResponse"
},
"403": {
"$ref": "#/responses/forbidden"
}
}
}
},
"/orgs/{org}/teams": {
"get": {
"produces": [
@ -15844,6 +16006,123 @@
}
}
},
"/repos/{owner}/{repo}/settings/actions/permissions": {
"get": {
"produces": [
"application/json"
],
"tags": [
"repository"
],
"summary": "Get repository Actions token permissions",
"operationId": "repoGetActionsPermissions",
"parameters": [
{
"type": "string",
"description": "owner of the repo",
"name": "owner",
"in": "path",
"required": true
},
{
"type": "string",
"description": "name of the repo",
"name": "repo",
"in": "path",
"required": true
}
],
"responses": {
"200": {
"$ref": "#/responses/ActionsPermissionsResponse"
},
"404": {
"$ref": "#/responses/notFound"
}
}
},
"put": {
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"repository"
],
"summary": "Update repository Actions token permissions",
"operationId": "repoUpdateActionsPermissions",
"parameters": [
{
"type": "string",
"description": "owner of the repo",
"name": "owner",
"in": "path",
"required": true
},
{
"type": "string",
"description": "name of the repo",
"name": "repo",
"in": "path",
"required": true
},
{
"name": "body",
"in": "body",
"schema": {
"$ref": "#/definitions/ActionsPermissions"
}
}
],
"responses": {
"200": {
"$ref": "#/responses/ActionsPermissionsResponse"
},
"403": {
"$ref": "#/responses/forbidden"
},
"422": {
"$ref": "#/responses/validationError"
}
}
},
"delete": {
"produces": [
"application/json"
],
"tags": [
"repository"
],
"summary": "Reset repository Actions permissions to default",
"operationId": "repoResetActionsPermissions",
"parameters": [
{
"type": "string",
"description": "owner of the repo",
"name": "owner",
"in": "path",
"required": true
},
{
"type": "string",
"description": "name of the repo",
"name": "repo",
"in": "path",
"required": true
}
],
"responses": {
"204": {
"$ref": "#/responses/empty"
},
"403": {
"$ref": "#/responses/forbidden"
}
}
}
},
"/repos/{owner}/{repo}/signing-key.gpg": {
"get": {
"produces": [

View File

@ -4,6 +4,7 @@
package migrations
import (
"bytes"
"compress/gzip"
"context"
"database/sql"
@ -21,7 +22,6 @@ import (
"code.gitea.io/gitea/models/migrations"
migrate_base "code.gitea.io/gitea/models/migrations/base"
"code.gitea.io/gitea/models/unittest"
"code.gitea.io/gitea/modules/charset"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
@ -108,11 +108,11 @@ func readSQLFromFile(version string) (string, error) {
}
defer gr.Close()
bytes, err := io.ReadAll(gr)
buf, err := io.ReadAll(gr)
if err != nil {
return "", err
}
return string(charset.MaybeRemoveBOM(bytes, charset.ConvertOpts{})), nil
return string(bytes.TrimPrefix(buf, []byte{'\xef', '\xbb', '\xbf'})), nil
}
func restoreOldDB(t *testing.T, version string) {

View File

@ -2,20 +2,10 @@
import {SvgIcon} from '../svg.ts';
import {isPlainClick} from '../utils/dom.ts';
import {shallowRef} from 'vue';
import {type createViewFileTreeStore} from './ViewFileTreeStore.ts';
export type Item = {
entryName: string;
entryMode: 'blob' | 'exec' | 'tree' | 'commit' | 'symlink' | 'unknown';
entryIcon: string;
entryIconOpen: string;
fullPath: string;
submoduleUrl?: string;
children?: Item[];
};
import type {createViewFileTreeStore, FileTreeItem} from './ViewFileTreeStore.ts';
const props = defineProps<{
item: Item,
item: FileTreeItem,
store: ReturnType<typeof createViewFileTreeStore>
}>();

View File

@ -3,11 +3,20 @@ import {GET} from '../modules/fetch.ts';
import {pathEscapeSegments} from '../utils/url.ts';
import {createElementFromHTML} from '../utils/dom.ts';
import {html} from '../utils/html.ts';
import type {Item} from './ViewFileTreeItem.vue';
export type FileTreeItem = {
entryName: string;
entryMode: 'blob' | 'exec' | 'tree' | 'commit' | 'symlink' | 'unknown';
entryIcon: string;
entryIconOpen: string;
fullPath: string;
submoduleUrl?: string;
children?: Array<FileTreeItem>;
};
export function createViewFileTreeStore(props: {repoLink: string, treePath: string, currentRefNameSubURL: string}) {
const store = reactive({
rootFiles: [] as Array<Item>,
rootFiles: [] as Array<FileTreeItem>,
selectedItem: props.treePath,
async loadChildren(treePath: string, subPath: string = '') {

View File

@ -12,8 +12,8 @@ declare module '*.vue' {
import type {DefineComponent} from 'vue';
const component: DefineComponent<unknown, unknown, any>;
export default component;
// List of named exports from vue components, used to make `tsc` output clean.
// To actually lint .vue files, `vue-tsc` is used because `tsc` can not parse them.
// Here we declare all exports from vue files so `tsc` or `tsgo` can work for
// non-vue files. To lint .vue files, `vue-tsc` must be used.
export function initDashboardRepoList(): void;
export function initRepositoryActionView(): void;
}