Skip to content

Commit c836b6e

Browse files
palikarpalikar
committed
Integrating the depender in the manager
1 parent 7804c4e commit c836b6e

9 files changed

Lines changed: 171 additions & 99 deletions

File tree

‎.pylintrc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ max-branches=20
88

99
[FORMAT]
1010
max-line-length = 120
11-
disable=locally-disabled,locally-enabled,missing-docstring,duplicate-code,fixme,cyclic-import,redefined-variable-type,stop-iteration-return, R1705, E1136, W0613
11+
disable=locally-disabled,locally-enabled,missing-docstring,duplicate-code,fixme,cyclic-import,redefined-variable-type,stop-iteration-return, R1705, E1136, W0613, R0902
1212

1313
[TYPECHECK]
1414

‎Makefile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ help:
4343
@echo 'make snapshot: Create a tar.gz of the current git revision'
4444
@echo 'make pypi_sdist: Release a new sdist to PyPI'
4545

46-
test: test_pylint test_flake8 test_pylint
46+
test: test_pylint test_flake8 test_pytest
4747
@echo "All test ran..."
4848

4949
test_pylint:
@@ -54,7 +54,7 @@ test_flake8:
5454
@echo "Running flake8..."
5555
flake8 $(TEST_PATHS)
5656

57-
test_pylint:
57+
test_pytest:
5858
@echo "Running pylint..."
5959
pytest
6060

‎README.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
1-
Build status: [![Build Status](https://travis-ci.org/palikar/code_manager.svg?branch=master)](https://travis-ci.org/palikar/code_manager)
1+
Build status:
2+
[![Build Status](https://travis-ci.org/palikar/code_manager.svg?branch=master)](https://travis-ci.org/palikar/code_manager)
3+
[![Build Status](https://pyup.io/repos/github/palikar/code_manager/shield.svg)](https://pyup.io/repos/github/palikar/code_manager)
4+
[![Build Status](https://pyup.io/repos/github/palikar/code_manager/python-3-shield.svg)](https://pyup.io/repos/github/palikar/code_manager/)
5+
[![Coverage Status](https://coveralls.io/repos/github/palikar/code_manager/badge.svg?branch=master)](https://coveralls.io/github/palikar/code_manager?branch=master)
26

37

48
# Code Manager
59

6-
![img](https://travis-ci.org/palikar/code_manager.svg?branch=master) ![img](https://pyup.io/repos/github/palikar/code_manager/shield.svg) ![img](https://pyup.io/repos/github/palikar/code_manager/python-3-shield.svg) ![img](https://coveralls.io/repos/github/palikar/code_manager/badge.svg?branch=master)
7-
810

911
## Abstract
1012

‎README.org‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,16 +15,19 @@
1515
#+CREATOR: Emacs 26.1 (Org mode 9.1.13)
1616

1717

18-
18+
#+BEGIN_EXAMPLE
19+
Build status:
20+
[![Build Status](https://travis-ci.org/palikar/code_manager.svg?branch=master)](https://travis-ci.org/palikar/code_manager)
21+
[![Build Status](https://pyup.io/repos/github/palikar/code_manager/shield.svg)](https://pyup.io/repos/github/palikar/code_manager)
22+
[![Build Status](https://pyup.io/repos/github/palikar/code_manager/python-3-shield.svg)](https://pyup.io/repos/github/palikar/code_manager/)
23+
[![Coverage Status](https://coveralls.io/repos/github/palikar/code_manager/badge.svg?branch=master)](https://coveralls.io/github/palikar/code_manager?branch=master)
24+
#+END_EXAMPLE
1925

2026
* Code Manager
2127

22-
![[https://travis-ci.org/palikar/code_manager.svg?branch=master][img]]
23-
![[https://pyup.io/repos/github/palikar/code_manager/shield.svg][img]]
24-
![[https://pyup.io/repos/github/palikar/code_manager/python-3-shield.svg][img]]
25-
![[https://coveralls.io/repos/github/palikar/code_manager/badge.svg?branch=master][img]]
2628

2729
** Abstract
30+
2831
This is my personal tool now for managing my github repositories, some system software that I use and pretty much everything that can be downloaded, compiled locally and then installed on a Debian based Linux system. Through this utility one can quickly download and install random things from all over the internet. I've always wanted some small program that would allow me to quickly bring my github repositories on my local machine so I end it up writing this in my spare time. The program is focused on automation but also on flexibility in the installation process. A lot of software is compiled and installed in some standard way but there are also things that are a little bit trickier. The utility - named appropriately ~code_manager~ - aims to provide a unified interface for the installation process of all types of software -- the trickier kind included.
2932

3033

@@ -33,6 +36,7 @@ Currently the project is not on [[https://pypi.org/][PyPi]] so you have to clone
3336
** Installation
3437

3538
The installation from source is possible through this repository.
39+
3640
#+BEGIN_SRC sh
3741
git clone https://github.com/palikar/code_manager
3842
cd code_manager

‎code_manager/core/debgrapher.py‎

Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,10 @@ def get_packages(self, group=""): # pylint: disable=R1705
2323

2424
def verify_package_list(self, package_names):
2525
all_packs = set(flatten(self.packages_list.values()))
26-
print(all_packs)
2726
for pack in package_names:
2827
if pack not in all_packs:
2928
logging.info("The package %s is not\
30-
in the list with packags.", pack)
29+
in the list with packags.", pack)
3130
logging.info("Did you mean any of the following: %s",
3231
",".join(difflib.get_close_matches(pack, all_packs, 5)))
3332
exit(1)
@@ -39,32 +38,40 @@ def verify_packages_tree(self):
3938
all_packs_list = set(flatten(self.packages_list.values()))
4039
all_packs_nodes = set(self.packages.keys()) # pylint: disable=E1101
4140

41+
broken = []
4242
# Every package is in a group
4343
for pack in all_packs_nodes:
4444
if pack not in all_packs_list:
45-
logging.critical("Incosistant packages file!\
46-
The package %s is not in any group", pack)
47-
exit(1)
45+
broken.append(pack)
46+
47+
if broken:
48+
logging.critical("Inconsistent packages file!\
49+
The packages [%s] are not in any group", ','.join(broken))
50+
exit(1)
4851

52+
broken = []
4953
# Every package in a group is in the nodes
5054
for pack in all_packs_list:
5155
if pack not in all_packs_nodes:
52-
logging.critical("Incosistant packages file!\
53-
The package %s does not have node", pack)
54-
exit(1)
56+
broken.append(pack)
57+
58+
if broken:
59+
logging.critical("Incosistant packages file!\
60+
The packages [%s] do not have node", ','.join(broken))
61+
exit(1)
5562

5663
# Every dependency is in the packages.json
5764
for pack in all_packs_nodes:
5865
for deb in self.get_dependencies(pack):
5966
if deb not in all_packs_nodes:
6067
logging.critical("%s is dependency\
61-
of %s but it is not in the packages.json", deb, pack)
68+
of %s but it is not in the packages.json", deb, pack)
6269
exit(1)
6370

6471
def get_dependencies(self, package):
6572
if package not in self.packages.keys(): # pylint: disable=E1101
6673
logging.critical("The package %s\
67-
is not in the packages.json.", package) # pylint: disable=E1136
74+
is not in the packages.json.", package) # pylint: disable=E1136
6875
exit(1)
6976
if 'dependencies' in self.packages[package].keys():
7077
return self.packages[package]['dependencies'] # pylint: disable=E1136
@@ -102,7 +109,7 @@ def get_build_order(self, packages):
102109
available = [pack for pack, deps in sub_tree.items() if not deps]
103110
if not available:
104111
logging.critical('Build order cannot be generated.\
105-
The packages tree is maybe broken.')
112+
The packages tree is maybe broken.')
106113
exit(1)
107114
for pack in available:
108115
build_order.append(pack)

‎code_manager/core/installation.py‎

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -76,28 +76,49 @@ def _add_installer(self, installer, file):
7676
def run_installer(self, name, installer):
7777

7878
if installer not in self.installers.keys():
79-
logging.critical('There is no installer with the name %s', name)
79+
logging.critical('There is no installer with the name %s', installer)
8080

8181
if installer not in self.installer_objects.keys():
8282
installer_obj = self.installers[installer]()
8383
self.installer_objects[installer] = installer_obj
8484
else:
8585
installer_obj = self.installer_objects[installer]
8686

87-
node = self.packages()[name]
87+
node = self.packages[name]
8888
installer_obj.node = node
8989

9090
if hasattr(installer_obj, 'manditory_attr') and isinstance(installer_obj.manditory_attr, list):
9191
for attr in installer_obj.manditory_attr:
9292
if attr not in node.keys():
9393
logging.critical('The attribute %s is mandatory for the installer %s\
94-
but it is not in the package node.',
95-
attr, installer_obj.name)
94+
but it is not in the package node of %s.',
95+
attr, installer_obj.name, name)
9696

9797
result = installer_obj.execute(name)
9898

99-
if result > 0:
100-
logging.critical('The installer %s failed to execute properly', installer_obj.nam)
99+
if result is None:
100+
logging.critical('The installer [%s] failed to execute properly', installer_obj.name)
101+
102+
def install(self, package):
103+
assert package is not None
104+
node = self.packages[package]
105+
106+
if 'install' not in node.keys():
107+
return 0
108+
109+
installer = node['install']
110+
if isinstance(installer, str):
111+
self.run_installer(package, installer)
112+
return 0
113+
elif isinstance(installer, list):
114+
for inst in installer:
115+
self.run_installer(package, inst)
116+
return 0
117+
else:
118+
logging.critical('Can\'t install %s.\
119+
Installation node is nor a list, nor a string.', package)
120+
exit(1)
121+
return None
101122

102123

103124
# class Installer:

‎code_manager/core/manager.py‎

Lines changed: 74 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77
from code_manager.core.debgrapher import DebGrapher
88
from code_manager.core.cache_container import CacheContainer
99
from code_manager.utils.utils import flatten
10-
from code_manager.utils.logger import debug_red
1110

1211

1312
class Manager(ConfigurationAware):
@@ -30,52 +29,82 @@ def __init__(self):
3029
def _setup_all(self):
3130
self.installation.load_installer()
3231
self.cache.load_cache()
32+
self.depender.verify_packages_tree()
3333

3434
def _invoke(self):
35-
pass
35+
logging.info('Invoking installation with: %s',
36+
','.join(self.install_queue))
37+
logging.info('Steps configuration:')
38+
logging.info('\tInstall:%s', self.install)
39+
logging.info('\tBuild:%s', self.build)
40+
logging.info('\tFetching:%s', self.fetching)
41+
42+
self.depender.verify_package_list(self.install_queue)
43+
44+
if self.install:
45+
self._invoke_install()
46+
47+
if self.fetching:
48+
self._invoke_fetch()
49+
50+
if self.fetching:
51+
self._invoke_build()
52+
53+
def _invoke_fetch(self):
54+
for pack in self.install_queue:
55+
with self.cache as cache:
56+
57+
if not cache.is_fetched(pack):
58+
if self.fetcher.download(pack, pack) is None:
59+
logging.critical("The fetching of '%s' failed.", pack)
60+
cache.set_fetched(pack, True)
61+
else:
62+
logging.info("\'%s\' is already fetched", pack)
3663

3764
def _invoke_build(self):
3865
pass
3966

4067
def _invoke_install(self):
41-
pass
42-
43-
def fetch_package(self, package):
44-
with self.cache as cache:
45-
if cache.is_fetched(package):
46-
debug_red("The package '%s' is already fetched", package)
47-
return None
48-
if self.fetcher.download(package, package) is None:
49-
debug_red("The fetching of '%s' failed.", package)
50-
return None
51-
else:
52-
cache.set_fetched(package, True)
53-
cache.set_root(package, package)
54-
return 0
55-
56-
def fetch_group(self, group):
57-
for pack in self.packages_list[group]:
58-
self.fetch_package(pack)
59-
60-
def fetch_thing(self, thing):
61-
if thing in self.packages_list.keys():
62-
logging.info("'%s' is a group. Fetching all packages in it.", thing)
63-
self.fetch_group(thing)
64-
elif thing in flatten(self.packages_list.values()):
65-
logging.info("'%s' is a package. Fetching it.", thing)
66-
self.fetch_package(thing)
67-
else:
68-
logging.info("There is no thing with name '%s'", thing)
69-
70-
def fetch(self, thing):
71-
if isinstance(thing, list):
72-
for thingy in thing:
73-
self.fetch_thing(thingy)
74-
elif isinstance(thing, str):
75-
self.fetch_thing(thing)
76-
else:
77-
logging.critical("Can't install %s. It's \
78-
no string nor list", thing)
68+
extended_queue = set(self.install_queue)
69+
for pack in self.install_queue:
70+
extended_queue.update(self.depender.get_deep_dependencies(pack))
71+
logging.debug('Extended queue: %s', ','.join(extended_queue))
72+
73+
ordered_packages = self.depender.get_build_order(list(extended_queue))
74+
logging.debug('Build order: %s', ','.join(ordered_packages))
75+
76+
self._check_install_nodes(ordered_packages)
77+
78+
for pack in ordered_packages:
79+
with self.cache as cache:
80+
81+
if not cache.is_fetched(pack):
82+
if self.fetcher.download(pack, pack) is None:
83+
logging.critical("The fetching of '%s' failed.", pack)
84+
cache.set_fetched(pack, True)
85+
else:
86+
logging.info("\'%s\' is already fetched", pack)
87+
88+
if not cache.is_installed(pack):
89+
if self.installation.install(pack) == 0:
90+
logging.info("\'%s\' was installed", pack)
91+
cache.set_installed(pack, True)
92+
else:
93+
logging.info("\'%s\' is already installed", pack)
94+
# TODO: Update\Build the package here
95+
96+
cache.set_built(pack, True)
97+
98+
def _check_install_nodes(self, packages):
99+
for pack in packages:
100+
pack_node = self.packages[pack]
101+
if 'install' not in pack_node.keys():
102+
continue
103+
installer = pack_node['install']
104+
if not isinstance(installer, str) and not isinstance(installer, list):
105+
logging.critical('Can\'t install %s.\
106+
Installation node is nor a list, nor a string.', pack)
107+
exit(1)
79108

80109
def _install_thing(self, thing):
81110

@@ -90,16 +119,16 @@ def _install_thing(self, thing):
90119
self.install_queue.append(thing)
91120

92121
else:
93-
logging.critical("There is no thing with name %s", thing)
122+
self.install_queue.append(thing)
94123

95124
def install_thing(self, thing, install=True, fetch=False, build=False):
96125
if install:
97126
self.install = True
98-
self.fetching = True
99-
self.build = True
127+
self.fetching = False
128+
self.build = False
100129
elif build:
101130
self.install = False
102-
self.fetching = True
131+
self.fetching = False
103132
self.build = True
104133
elif fetch:
105134
self.install = False
@@ -113,5 +142,5 @@ def install_thing(self, thing, install=True, fetch=False, build=False):
113142
self._install_thing(thing)
114143

115144
if self.install_queue:
116-
self._setup_all()
145+
# self._setup_all()
117146
self._invoke()

‎code_manager/installers/command.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,10 @@ def __init__(self):
1010
BasicInstaller.__init__(self)
1111

1212
def execute(self, name):
13-
pass
13+
return 0
1414

1515
def update(self, name):
16-
pass
16+
return 0
1717

1818

1919
ExportedClass = CommandInstaller

0 commit comments

Comments
 (0)