Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Cod
10-20-2013, 06:57 AM (This post was last modified: 03-13-2017 08:18 PM by lingu.)
Post: #1
Cod
Cod is a Python script that helps retrieve source and binary files from multiple repositories of potentially different types. It considers all world's program code, source and binary, as a forest, and helps people walk and work in this forest.

You can think of Cod as a meta source code version control system. It enables the programmer to manage a "source forest", not a source tree. Given a module name, context or other relevant information, cod attempts to direct the actions to specific repositories at different locations. It restricts the actions upon a repository to enforce a simple workflow.

Currently cod supports git, and, consequently, you can use cod like git, but remember that this will change soon and cod restricts the actions to a subset of git's functions.

Prerequisites
Cod typically runs in a command-line interface, such as a shell in or a Cygwin command window on Microsoft Windows.

Cod requires the following software:
1. SSH
2. Python (cod assumes the python program is in /usr/bin/ )
3. git (client)

How to get cod
Installing cod is easy -- you simply get the cod program and run it. The cod program is distributed in the following two ways.

(1) Python source code: The source code of cod is available at git://codb.d-thinker.org/code.git and you can obtain a read-only copy of the source code with the following command:

Code:
git clone git://codb.d-thinker.org/code

Note that the repository name is 'code', not 'cod'. The cod program is cod.py in the repository, and you can run it directly in a Python-enabled environment.

For advanced users: You need to also set up authentication keys and Cod extensions (to be introduced in detail later). Refer to the 'Authentication' section for instructions on setting up keys. The extension files is usually set up automatically by cod when you run a cod command, and you can add more modules to the extension after you get familiar with how it works.

For basic set up, you can run the 'update' command of cod to set up the cod working environment, as shown in the following command line.

Code:
python code/cod.py update
python code/cod.py update

Note that you need to run 'update' twice. This is due to a special situation in the automatic extension setup process. In the future, you need to run 'cod update' once to update all the code trees in the extension.

Note that by default, repositories cloned with the above command is read-only.

Completing these steps, you have set up the cod tool! You may skip the following sections and start using cod. Later, when you want to do advanced configurations, you can come back to this article and continue with further configurations. Enjoy!

For advanced users: You might want to commit changes to the repository as a developer. In that case, you can refer to the "Update read-only repository configuration with update_local" session in later parts of this tutorial. You can also clone a repository as an authenticated user from the start, please refer to the session "Clone a repository as an authenticated user".

Now cod is set up in the right directory, most likely ~/forest/code/code, and you can starting using it. If the command above fails due to authentication errors, most likely the repository is not open to public, and authentication is needed. Please refer to the section "Authentication" below to set up the keys.

To learn more about the cod command line, use the following command to print the help message.

Code:
python code/cod.py help

There is also a shell script 'cod' in the code directory that helps run the cod.py python script. Therefore, the command above is equivalent to

Code:
code/cod help

which is hopefully more convenient.

(2) (Temporarily not available) Standalone program: Pre-compiled programs are available for downloading at http://cod.shufangkeji.com/ . You can download a suitable version and run it directly on Linux. This currently works only in the Linux environment.

Note that the "git clone" command placed the cod programs in ~/code. This is different from the normal convention of cod. Usually, cod stores code trees in the $GROUND_DIR directory, which is usually "~/forest". Since you have already got the cod program, you can obtain itself again by running the following command.

Code:
code/cod read code

This will store the cod programs in $GROUNND_DIR/code/code. Be aware that 'cod read' obtains a read-only copy of the program. If you want to modify the cod program, you need to use 'cod clone' instead of 'cod read'.

It is advisable to include "cod" in a directory in PATH so that you can directly run it in the command line. For example, if you use bash, you may want to add the following line to the ~/.bashrc

Code:
export PATH=$PATH:~/forest/code/code

You may also choose to create an alias for invoking "python /path/to/cod.py".

After cod is functional, you can always update to the latest version of cod using cod itself. Examples in this post provide hints on how to do it.

For advanced users: Authentication

The "cod read" command we described earlier does not require authentication. However, if you like to have write-access to some code trees, it is necessary to set up authentication. Cod uses public/private key authentication. New users need to generate a key pair if they don’t have one, and send the public key to the administrator. The user can generate the key pair by executing this command:

Code:
$ ssh-keygen -t rsa

After generating the key pair, send the public key (~/.ssh/id_rsa.pub) to the administrator and ask he/she to create an account for you. After receiving confirmation from the administrator, the user can starts to use cod.

If the private key corresponding to the public key you send to the administrator is not the default one (~/.ssh/id_rsa) used by SSH, the user needs to direct SSH to use the matching private key when running Cod. One solution is to use the config file ~/.ssh/config and add a few lines like the following example. Suppose the private key to be used for cod is id_rsa_codb.

Code:
Host codb.d-thinker.org
Port 22
User git
IdentityFile ~/.ssh/id_rsa_codb

Using the default key pair for Cod will eliminate this complication. Nevertheless, you are advised to add a few entries to ~/.ssh/config because some code trees use non-standard SSH ports. Please refer to http://tab.d-thinker.org/showthread.php?tid=2581 for details.

For advanced users: Update configuration with update_local
An existing repository can be read-only without authentication. The config file of the repository has url in the form of "git://....". You might want to update the code and commit the changes, in that case, go to the repository folder, and use this command.
Code:
cod update_local

The command modify the config file for git and you can now commit to the repository if you are an authenticated user.

For advanced users: Clone a repository as an authenticated user
You can also clone a repository ( i.e., the repository does not exists in the forest so far ) as an authenticated user. In such case, you can make modifications and commit changes.
Code:
cod clone <repository name>
If the command fails, most likely you are not authenticated to the repository. Please contact the administrator in this case.

Usage
Cod manages a "source forest", and the forest contains one or more source trees, each tree managed by an version control system, such as git (Subversion will be supported soon). It is supposed that all the source code trees are in one "ground" directory. Usually the ground directory is the directory in which you issue "cod clone" commands. Suppose the ground directory's name is "shenandoah", the source code trees may look like the following example:


shenandoah
|___ tree1 (git repository one)
|___ tree2 (git repository two)
|___ tree3 (git repository three)


But the directories should not form a structure like the following:


shenandoah
|___ tree1 (git repository one)
|___ subdir2 ___ subdir22 ___ tree2 (git repository two)
|___ subdir3 ___ tree3 (git repository three)


That is to say, the forest ground directory should contain only sub-directorys that represents a valid source code tree managed by some version control system. In practice, this requirement can be easily met by issuing the source tree retrieval command (git clone, svn co, etc.) in the ground directory.

The following examples show typical usage of cod.

- To print the list of cod commands, run

cod help

- The cod standalone program itself is contained in the 'bin' code tree. You can clone the bin code tree as follows and obtain the latest version of cod:

cod clone bin

Clone should be done in the ground directory. If the clone is successful, two directories 'bin' and 'bin/bin' should be created and contain the file cod together with other files in the code tree.

- Many cod commands need to run in the work directory managed by the version management tool (e.g., git). We should thus enter the work directory as shown in the following example.

cd $GROUND_DIR/bin/bin

- We typically use branches to manage multiple concurrent versions of the code. Refer to Branch rules for more information. For example, if we work on the 'rc' branch, we can use the following git command to switch to the required branch.

git checkout rc

- To pull the latest files from the repository to the local cloned source tree, run

cod pull

- Create and add a file. For example, add a file README as follows.

cod add README

- Edit files and commit the changes. For example, run the following command to commit changes in README.

cod commit -m "Some modifications on README" README

- Push the changes to the server.

cod push

Extensions
Many developers want to extend cod for his or her own environment. Cod has an extension interface that allows the developer to do so -- http://tab.shufangkeji.com:680/showthread.php?tid=2424 .

---
Revisions:
20170206/zli: Add one sentence to denote the end of simple set up
20170106/zli: Revert changes about two "cod update" because it's still needed.
20170105/zli: Add "update_local" and "clone" session for advanced users
20170105/zli: Remove authentication requirement, remove the requirement for "update" twice
20160524/gl: State that the standalone version is not available now.
20150616/gl: Mention the key problem if cod update fails.
20150616/gl: Recommend to use 'cod update' after getting cod.
20150314/gl: Remove the .hope file usage.
20141203/gl: Update the URL for standalone distributions to cod.shufangkeji.com. Remove the attachment (cod.py) in this post.
20141127/gl: Update the URL cod.d-thinker.org.
20141122/gl: Introduce the code/cod script.
20141018/gl: Add a link to specific configurations in ~/.ssh/config.
20140823/gl: Add a section for extensions.
20140822/gl: Add git info.
Quote this message in a reply
03-29-2014, 02:29 AM
Post: #2
RE: Cod
(10-20-2013 06:57 AM)lingu Wrote:  How to get cod
There isn't a really need for "installing cod". Cod is distributed as a standalone program and you can directly download it from http://www.zettadom.com/tool/cod and run it on Linux.

The download link is updated to http://ada.zettadom.com:1220/test.html
Quote this message in a reply
03-30-2014, 05:36 AM
Post: #3
RE: Cod
(03-29-2014 02:29 AM)DaY Wrote:  [quote='lingu' pid='5841' dateline='1382223440']
How to get cod
The download link is updated to http://ada.zettadom.com:1220/test.html
Now, test.html is renamed to cod.html, so the download link should be http://ada.zettadom.com:1220/cod.html
Quote this message in a reply
08-24-2014, 12:07 AM
Post: #4
RE: Cod
A new command 'get' is added. 'cod get' retrieves a read-only copy of a code tree if the code tree allows public accesses.
Find all posts by this user
Quote this message in a reply
08-26-2014, 11:32 PM
Post: #5
[Bug report] problem cloning 2 repos
When cloning 2 repos using cod, the second repo's name turns to be one with a ':' added.

$ cod.py clone l0-scheduler libi0
The Ground is the directory where all the code trees reside.
The code tree will be cloned in the current Ground: /tmp/tg
Cloning into 'l0-scheduler'...
remote: Counting objects: 340, done.
remote: Compressing objects: 100% (328/328), done.
Receiving objects: 100% (340/340), 85.54 KiB | 0 bytes/s, done.
remote: Total 340 (delta 221), reused 0 (delta 0)
Resolving deltas: 100% (221/221), done.
Checking connectivity... done.
Cloning into ':libi0'... <------------------- bug here
remote: Counting objects: 368, done.
remote: Total 368 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (368/368), 733.82 KiB | 292.00 KiB/s, done.
Resolving deltas: 100% (244/244), done.
Checking connectivity... done.
Visit this user's website Find all posts by this user
Quote this message in a reply
08-27-2014, 03:05 AM
Post: #6
Fixed in b12c5f67279306b5baab261c9f20d0c4c206fd42
(08-26-2014 11:32 PM)zma Wrote:  When cloning 2 repos using cod, the second repo's name turns to be one with a ':' added.

$ cod.py clone l0-scheduler libi0
The Ground is the directory where all the code trees reside.
The code tree will be cloned in the current Ground: /tmp/tg
Cloning into 'l0-scheduler'...
remote: Counting objects: 340, done.
remote: Compressing objects: 100% (328/328), done.
Receiving objects: 100% (340/340), 85.54 KiB | 0 bytes/s, done.
remote: Total 340 (delta 221), reused 0 (delta 0)
Resolving deltas: 100% (221/221), done.
Checking connectivity... done.
Cloning into ':libi0'... <------------------- bug here
remote: Counting objects: 368, done.
remote: Total 368 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (368/368), 733.82 KiB | 292.00 KiB/s, done.
Resolving deltas: 100% (244/244), done.
Checking connectivity... done.

It's fixed in code commit b12c5f67279306b5baab261c9f20d0c4c206fd42.

The reason is a problematic access string for https accesses.
Find all posts by this user
Quote this message in a reply
08-30-2014, 12:19 AM
Post: #7
[Bug report] Error in importing CodExt
[zma@ust:~/forest/code/code]$ git branch
master
* rc
[zma@ust:~/forest/code/code]$ git log -1
commit b12c5f67279306b5baab261c9f20d0c4c206fd42
Author: gl <lingu@ieee.org>
Date: Tue Aug 26 14:40:10 2014 +0800

Fix https access string
[zma@ust:~/forest/code/code]$ ./cod.py
Adding /home/zma/.cod to sys.path
Importing CodConfig
module_ctns_vec is
['bin']
Adding to sys.path /home/zma/forest/bin/bin
Importing CodExt
Error in importing CodExt

Cod command not recognized. Usage:

cod {status|pull|get|newme|clone|sync|update|add|install|rm|push|commit|history} {-m <log message>} <module_name>
Visit this user's website Find all posts by this user
Quote this message in a reply
08-30-2014, 02:26 AM
Post: #8
Fixed with less verbose message printing
(08-30-2014 12:19 AM)zma Wrote:  [zma@ust:~/forest/code/code]$ git branch
master
* rc
[zma@ust:~/forest/code/code]$ git log -1
commit b12c5f67279306b5baab261c9f20d0c4c206fd42
Author: gl <lingu@ieee.org>
Date: Tue Aug 26 14:40:10 2014 +0800

Fix https access string
[zma@ust:~/forest/code/code]$ ./cod.py
Adding /home/zma/.cod to sys.path
Importing CodConfig
module_ctns_vec is
['bin']
Adding to sys.path /home/zma/forest/bin/bin
Importing CodExt
Error in importing CodExt

Cod command not recognized. Usage:

It is a harmless debug message. I've disabled verbose printing so that it is not printed for normal use.
Find all posts by this user
Quote this message in a reply
09-20-2014, 04:41 PM
Post: #9
RE: Cod
cod's previous download link disabled, please download from http://115.28.77.20/
Quote this message in a reply
10-21-2014, 02:57 PM
Post: #10
RE: Cod
Currently, before you use cod.py, you have to config ~/.cod/CodConfig otherwise you would get following errors. (To add cod modules, edit ~/.cod/CodConfig to include additional code tree names for the modules. The first module may be an extension
of Cod.)

harryxiyou@common_vm ~/forest $ cod.py --help
Traceback (most recent call last):
File "/lhome/harryxiyou/forest/code/code/cod.py", line 82, in <module>
readModuleNames()
File "/lhome/harryxiyou/forest/code/code/cod.py", line 73, in readModuleNames
module_ctns_vec = CodConfig.ExtensionCTNs
AttributeError: 'module' object has no attribute 'ExtensionCTNs'


*Config it as follows*

harryxiyou@common_vm ~/forest $ cat ~/.cod/CodConfig.py
# Define a list of names of code trees that contain Cod extensions.
ExtensionCTNs = ['bin' 'code' 'bin-obs' 'sqle' 'kvclient' 'cc0' 'libi0' 'zyzzyva' 'kvplus-obs' 'kvplus' 'tc2' 'kvpi' 'dc-programming' 'codb2-mgmt' 'dc-network-sensor' 'baijia-mgmt' 'forum' 'nodelist' 'codb-mgmt' 'git-control' 'l0sys' 'l0apps' 'vmr' 'l0-binary-translator' 'l0-scheduler' 'trajan-mgmt' 'vrs2' 'vman' 'localmem' 'puc']

# Add customerized code tree definitions in the following dict.
dctLocalCodeTreeDefs = {
"bin": {"proto":"git",
"username": "git",
"repository": "bin",
"loc": "codb2.lazero.net"},

"code": {"proto":"git",
"username": "git",
"repository": "code",
"loc": "codb.d-thinker.org"},
"bin-obs": {"proto":"git",
"username": "git",
"repository": "bin",
"loc": "codb.d-thinker.org"},
"sqle": {"proto":"git",
"username": "git",
"repository": "sqle",
"loc": "codb.d-thinker.org"},
"kvclient": {"proto":"git",
"username": "git",
"repository": "kvclient",
"loc": "codb.d-thinker.org"},
# obsolete since 20130901
#"cc0": {"proto":"git",
# "username": "git",
# "repository": "cc0",
# "loc": "git.baijia.info"},
"cc0": {"proto":"git",
"username": "",
"repository": "cc0-code",
"loc": "git://git.code.sf.net/p/cc0/code/ "},
"libi0": {"proto":"https",
"username": "",
"repository": "libi0",
# "loc": "gitgithub.com:layerzero/libi0.git "},
"loc": "github.com/layerzero/"},
"zyzzyva": {"proto":"git",
"username": "git",
"repository": "zyzzyva",
"loc": "git.baijia.info"},
# Migrated to codb2. 20140602/gl
"kvplus-obs": {"proto":"git",
"username": "git",
"repository": "kvplus",
"loc": "git.baijia.info"},
"kvplus": {"proto":"git",
"username": "git",
"repository": "kvplus",
"loc": "codb2.d-thinker.org"},
"tc2": {"proto":"git",
"repository": "tc2",
"username": "gito",
"loc": "codb3.datathinking.org"},
"kvpi": {"proto":"git",
"username": "git",
"repository": "kvplus-index",
"loc": "git.baijia.info"},
"dc-system": {"proto":"git",
"username": "git",
"repository": "dc-system",
"loc": "git.baijia.info"},
"dc-programming": {"proto":"git",
"username": "git",
"repository": "dc-programming",
"loc": "git.baijia.info"},
"codb2-mgmt": {"proto":"git",
"repository": "gitolite-admin",
"username": "git",
"loc": "codb2.d-thinker.org"},
"dc-network-sensor": {"proto":"git",
"username": "git",
"repository": "dc-network-sensor",
"loc": "git.baijia.info"},
"baijia-mgmt": {"proto":"git",
"repository": "gitosis-admin",
"username": "git",
"loc": "git.baijia.info"},
"forum": {"proto":"git",
"repository": "MYBB",
"username": "git",
"loc": "git.baijia.info"},
"nodelist": {"proto":"git",
"repository": "web-based-inventory-list",
"username": "git",
"loc": "git.baijia.info"},
"codb-mgmt": {"proto":"git",
"repository": "gitosis-admin",
"username": "git",
"loc": "codb.d-thinker.org"},
"git-control": {"proto":"git",
"username": "git",
"repository": "git-commit-email-prog",
"loc": "codb2.d-thinker.org"},
"l0sys": {"proto":"git",
"repository": "l0sys",
"username": "git",
"loc": "codb.d-thinker.org"},
# l0scheduler is likely obsolete
# "l0scheduler": {"proto":"git",
# "repository": "l0scheduler",
# "username": "git",
# "loc": "codb.lazero.net"},
"l0apps": {"proto":"git",
"repository": "l0apps",
"username": "git",
"loc": "codb.d-thinker.org"},
# disabled since 20140304
# "lib0": {"proto":"git",
# "repository": "lib0",
# "username": "git",
# "loc": "codb.lazero.net"},
"vmr": {"proto":"git",
"repository": "vmr",
"username": "git",
"loc": "codb.d-thinker.org"},
"l0-binary-translator": {"proto": "git",
"username": "git",
"repository": "l0-bin-translator",
"loc": "git.baijia.info:"},
"l0-scheduler": {"proto": "git",
"username": "git",
"repository": "l0-scheduler",
"loc": "trajan.lazero.net"},
"trajan-mgmt": {"proto":"git",
"repository": "gitolite-admin",
"username": "git",
"loc": "trajan.lazero.net"},
"vrs2": {"proto":"git",
"repository": "vrs2",
"username": "git",
"loc": "codb2.d-thinker.org"},
"vman": {"proto":"git",
"repository": "vman",
"username": "git",
"loc": "codb2.d-thinker.org"},
"localmem": {"proto":"git",
"repository": "localmem",
"username": "git",
"loc": "codb2.d-thinker.org"},
"puc": {"proto":"git",
"repository": "puc",
"username": "git",
"loc": "codb2.d-thinker.org"}
}
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: